Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Introduce PTEFile class - #15800

Merged
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head
Nov 18, 2025
Merged

Introduce PTEFile class#15800
meta-codesync[bot] merged 9 commits into
gh/lucylq/125/basefrom
gh/lucylq/125/head

Conversation

@lucylq

@lucylqlucylq commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
@pytorch-bot

pytorch-botBot commented Nov 13, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2025
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
Comment threadbackends/cadence/runtime/runtime.py

@zingozingo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@lucylq

Copy link
Copy Markdown
ContributorAuthor

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

@zingo

Copy link
Copy Markdown
Collaborator

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 14, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 322859321
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

  • Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
  • Refactors segment restoration logic by extracting _restore_constant_segment() function
  • Updates deserialize_pte_binary() return type from Program to PTEFile
  • Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
exir/_serialize/_program.pyCore implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.pyExports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.pyUpdates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.pyUpdates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.pyUpdates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.pyUpdates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.pyUpdates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.pyUpdates to access .program from nested deserialize_pte_binary() result

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py Outdated
Comment threadexir/_serialize/_program.py
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323865077
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
[ghstack-poisoned]
lucylq added a commit that referenced this pull request Nov 17, 2025
Pull Request resolved: #15800
PTEFile class holds the components of a PTE file: the program, mutable constants and named data.
Currently, the `program` definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the `program` concept.
Now, segment data is included in the PTEFile class.
ghstack-source-id: 323871061
@exported-using-ghexport
Differential Revision: [D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@meta-codesync
meta-codesyncBot merged commit e6c3a01 into gh/lucylq/125/baseNov 18, 2025
167 of 180 checks passed
@meta-codesync
meta-codesyncBot deleted the gh/lucylq/125/head branch November 18, 2025 03:09
@pytorchbotpytorchbot mentioned this pull request Nov 18, 2025
lucylq added a commit that referenced this pull request Nov 18, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: #15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
This PR was created by the merge bot to help merge the original PR into
the main branch.
ghstack PR number: pytorch#15800 by
@lucylq
^ Please use this as the source of truth for the PR details, comments,
and reviews
ghstack PR base:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/base
ghstack PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head:
https://github.com/pytorch/executorch/tree/gh/lucylq/125/orig
Differential Revision:
[D86814175](https://our.internmc.facebook.com/intern/diff/D86814175/)
@diff-train-skip-merge
Co-authored-by: lucylq <lfq@meta.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedmeta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@lucylq@zingo@larryliu0820