Uh oh!
There was an error while loading. Please reload this page.
RPATH Fix for portable_lib Python Extension - #13254
Merged
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13254
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 22 PendingAs of commit af215ba with merge base de24489 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
mergennachinforce-pushed
the
0.7_release_ready_pip_wheel_fix
branch
from
August 9, 2025 08:42
45f821c to
4230024Comparemergennachin
commented
Aug 9, 2025
ContributorAuthor
Note: if it's successful, make sure to merge to main as well |
Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues.
mergennachinforce-pushed
the
0.7_release_ready_pip_wheel_fix
branch
from
August 9, 2025 09:14
4230024 to
af215baCompareUh oh!
There was an error while loading. Please reload this page.
larryliu0820force-pushed
the
0.7_release_ready_pip_wheel_fix
branch
from
August 9, 2025 22:26
0793348 to
af215baComparelarryliu0820
approved these changes
Aug 9, 2025
Uh oh!
There was an error while loading. Please reload this page.
mergennachin
commented
Aug 11, 2025
ContributorAuthor
@pytorchbot cherry-pick --onto main -c critical |
pytorchbot pushed a commit
that referenced
this pull request
Aug 11, 2025
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find
PyTorch libraries when installed locally because it had hardcoded
absolute paths from
the CI build environment.
Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such
file)
Root Cause: The CMake build was linking to PyTorch libraries using
absolute paths from the build environment, without setting proper
relative RPATHs for
runtime library resolution.
Solution: Added platform-specific relative RPATH settings to the
portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines
657-669):
- macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries
relative to the .so file location
- Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
- Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency
Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch
libraries regardless of the installation location, fixing the runtime
linking issue
when using ExecutorTorch wheels built on CI.
Note: The same fix may be needed for _training_lib if it experiences
similar issues.
Test Plan:
```
# Build the wheel locally
python setup.py bdist_wheel
# create fresh conda env
conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11
# install
pip install ./dist/executorch-*.whl
# Verify
python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"
```
(cherry picked from commit 38062f8)pytorchbot
commented
Aug 11, 2025
Collaborator
Cherry picking #13254The cherry pick PR is at #13290 and it is recommended to link a critical cherry pick PR with an issue. Details for Dev Infra teamRaised by workflow job |
GregoryComer pushed a commit
to GregoryComer/executorch
that referenced
this pull request
Sep 18, 2025
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find
PyTorch libraries when installed locally because it had hardcoded
absolute paths from
the CI build environment.
Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such
file)
Root Cause: The CMake build was linking to PyTorch libraries using
absolute paths from the build environment, without setting proper
relative RPATHs for
runtime library resolution.
Solution: Added platform-specific relative RPATH settings to the
portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines
657-669):
- macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries
relative to the .so file location
- Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
- Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency
Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch
libraries regardless of the installation location, fixing the runtime
linking issue
when using ExecutorTorch wheels built on CI.
Note: The same fix may be needed for _training_lib if it experiences
similar issues.
Test Plan:
```
# Build the wheel locally
python setup.py bdist_wheel
# create fresh conda env
conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11
# install
pip install ./dist/executorch-*.whl
# Verify
python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"
```GregoryComer pushed a commit
to GregoryComer/executorch
that referenced
this pull request
Sep 19, 2025
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find
PyTorch libraries when installed locally because it had hardcoded
absolute paths from
the CI build environment.
Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such
file)
Root Cause: The CMake build was linking to PyTorch libraries using
absolute paths from the build environment, without setting proper
relative RPATHs for
runtime library resolution.
Solution: Added platform-specific relative RPATH settings to the
portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines
657-669):
- macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries
relative to the .so file location
- Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
- Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency
Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch
libraries regardless of the installation location, fixing the runtime
linking issue
when using ExecutorTorch wheels built on CI.
Note: The same fix may be needed for _training_lib if it experiences
similar issues.
Test Plan:
```
# Build the wheel locally
python setup.py bdist_wheel
# create fresh conda env
conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11
# install
pip install ./dist/executorch-*.whl
# Verify
python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"
```GregoryComer pushed a commit
to GregoryComer/executorch
that referenced
this pull request
Sep 19, 2025
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find
PyTorch libraries when installed locally because it had hardcoded
absolute paths from
the CI build environment.
Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such
file)
Root Cause: The CMake build was linking to PyTorch libraries using
absolute paths from the build environment, without setting proper
relative RPATHs for
runtime library resolution.
Solution: Added platform-specific relative RPATH settings to the
portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines
657-669):
- macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries
relative to the .so file location
- Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
- Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency
Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch
libraries regardless of the installation location, fixing the runtime
linking issue
when using ExecutorTorch wheels built on CI.
Note: The same fix may be needed for _training_lib if it experiences
similar issues.
Test Plan:
```
# Build the wheel locally
python setup.py bdist_wheel
# create fresh conda env
conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11
# install
pip install ./dist/executorch-*.whl
# Verify
python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"
```GregoryComer added a commit
that referenced
this pull request
Sep 19, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254 onto main. Fixes#14421. Original description below.** Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues. Test Plan: ``` # Build the wheel locally python setup.py bdist_wheel # create fresh conda env conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11 # install pip install ./dist/executorch-*.whl # Verify python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')" ``` Co-authored-by: Mergen Nachin <mnachin@meta.com>
pytorchbot pushed a commit
that referenced
this pull request
Sep 19, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254 onto main. Fixes#14421. Original description below.** Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues. Test Plan: ``` # Build the wheel locally python setup.py bdist_wheel # create fresh conda env conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11 # install pip install ./dist/executorch-*.whl # Verify python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')" ``` Co-authored-by: Mergen Nachin <mnachin@meta.com> (cherry picked from commit 641e737)
abhinaykukkadapu pushed a commit
that referenced
this pull request
Sep 23, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254 onto main. Fixes#14421. Original description below.** Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues. Test Plan: ``` # Build the wheel locally python setup.py bdist_wheel # create fresh conda env conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11 # install pip install ./dist/executorch-*.whl # Verify python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')" ``` Co-authored-by: Mergen Nachin <mnachin@meta.com> (cherry picked from commit 641e737)
abhinaykukkadapu pushed a commit
that referenced
this pull request
Sep 23, 2025
StrycekSimon pushed a commit
to nxp-upstream/executorch
that referenced
this pull request
Sep 23, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from pytorch#13254 onto main. Fixespytorch#14421. Original description below.** Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues. Test Plan: ``` # Build the wheel locally python setup.py bdist_wheel # create fresh conda env conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11 # install pip install ./dist/executorch-*.whl # Verify python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')" ``` Co-authored-by: Mergen Nachin <mnachin@meta.com>
jirioc pushed a commit
to nxp-upstream/executorch
that referenced
this pull request
Dec 19, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from pytorch#13254 onto main. Fixespytorch#14421. Original description below.** Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues. Test Plan: ``` # Build the wheel locally python setup.py bdist_wheel # create fresh conda env conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11 # install pip install ./dist/executorch-*.whl # Verify python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')" ``` Co-authored-by: Mergen Nachin <mnachin@meta.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from
the CI build environment.
Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file)
Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for
runtime library resolution.
Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669):
Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue
when using ExecutorTorch wheels built on CI.
Note: The same fix may be needed for _training_lib if it experiences similar issues.
Test Plan: