From 0715d3247262d03f354a4a89b5d6a02f6e6f57c6 Mon Sep 17 00:00:00 2001 From: Gregory Comer Date: Fri, 21 Feb 2025 00:37:54 -0800 Subject: [PATCH 1/7] Clean up getting-started.md, remove placeholders --- docs/source/backends-overview.md | 1 + docs/source/getting-started.md | 46 ++++++++++++++++---------------- 2 files changed, 24 insertions(+), 23 deletions(-) create mode 100644 docs/source/backends-overview.md diff --git a/docs/source/backends-overview.md b/docs/source/backends-overview.md new file mode 100644 index 00000000000..7b86b8bae7c --- /dev/null +++ b/docs/source/backends-overview.md @@ -0,0 +1 @@ +# Backend Overview \ No newline at end of file diff --git a/docs/source/getting-started.md b/docs/source/getting-started.md index e099eb98f63..24e37b5e9da 100644 --- a/docs/source/getting-started.md +++ b/docs/source/getting-started.md @@ -14,7 +14,7 @@ Pip is the recommended way to install the ExecuTorch python package. This packag pip install executorch ``` -To build the framework from source, see [Building From Source](TODO). +To build the framework from source, see [Building From Source](using-executorch-building-from-source.md). Backend delegates may require additional dependencies. See the appropriate backend documentation for more information. @@ -29,7 +29,7 @@ The following are required to install the ExecuTorch host libraries, needed to e
## Preparing the Model -Exporting is the process of taking a PyTorch model and converting it to the .pte file format used by the ExecuTorch runtime. This is done using Python APIs. PTE files for common models can be found on HuggingFace (TODO add link). +Exporting is the process of taking a PyTorch model and converting it to the .pte file format used by the ExecuTorch runtime. This is done using Python APIs. PTE files for common models, such as Llama 3, can be found on HuggingFace under [ExecuTorch Community](https://huggingface.co/executorch-community). These models have been exported and lowered for ExecuTorch, and can be directly deployed without needing to go through the lowering process. ### Requirements - A PyTorch model. @@ -39,7 +39,7 @@ Exporting is the process of taking a PyTorch model and converting it to the .pte ### Selecting a Backend ExecuTorch provides hardware acceleration for a wide variety of hardware. The most commonly used backends are XNNPACK, for Arm and x86 CPU, Core ML (for iOS), Vulkan (for Android GPUs), and Qualcomm (for Qualcomm-powered Android phones). -For mobile use cases, consider using XNNPACK for Android and Core ML or XNNPACK for iOS as a first step. See [Delegates](/TODO.md) for a description of available backends. +For mobile use cases, consider using XNNPACK for Android and Core ML or XNNPACK for iOS as a first step. See [Hardware Backends](using-executorch-export.md#hardware-backends) for more information. ### Exporting Exporting is done using Python APIs. ExecuTorch provides a high degree of customization during the export process, but the typical flow is as follows: @@ -96,7 +96,7 @@ Quick Links: #### Installation ExecuTorch provides Java bindings for Android usage, which can be consumed from both Java and Kotlin. -To add the library to your app, download the AAR, and add it to the gradle build rule. TODO Replace with Maven/Gradle package management when available. +To add the library to your app, download the AAR, and add it to the gradle build rule. ``` mkdir -p app/libs @@ -113,39 +113,39 @@ dependencies { #### Runtime APIs Models can be loaded and run using the `Module` class: ```java -import org.pytorch.executorch.EValue -import org.pytorch.executorch.Module -import org.pytorch.executorch.Tensor +import org.pytorch.executorch.EValue; +import org.pytorch.executorch.Module; +import org.pytorch.executorch.Tensor; // … Module model = Module.load(“/path/to/model.pte”) -// TODO Add input setup -EValue output = model.forward(input_evalue); + +Tensor input_tensor = Tensor.fromBlob(float_data, new long[] { 1, 3, height, width }); +EValue input_evalue = EValue.from(input_tensor); +EValue[] output = model.forward(input_evalue); +float[] scores = output[0].toTensor().getDataAsFloatArray(); ``` -For more information on Android development, including building from source, a full description of the Java APIs, and information on using ExecuTorch from Android native code, see [Using ExecuTorch on Android](/TODO.md). +For a full example of running a model on Android, see the [ExecuTorch Android Demo App](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/ExecuTorchDemo/app/src/main/java/com/example/executorchdemo/ClassificationActivity.java). For more information on Android development, including building from source, a full description of the Java APIs, and information on using ExecuTorch from Android native code, see [Using ExecuTorch on Android](using-executorch-android.md). ### iOS #### Installation -ExecuTorch supports both iOS and MacOS via C++ and Objective-C bindings, as well as hardware backends for CoreML, MPS, and CPU. The iOS runtime library is provided as a collection of .xcframework targets and are made available as a Swift PM package. +ExecuTorch supports both iOS and MacOS via C++, as well as hardware backends for CoreML, MPS, and CPU. The iOS runtime library is provided as a collection of .xcframework targets and are made available as a Swift PM package. -To get started with Xcode, go to File > Add Package Dependencies. Paste the URL of the ExecuTorch repo into the search bar and select it. Make sure to change the branch name to the desired ExecuTorch version in format “swiftpm-”, (e.g. “swiftpm-0.5.0”). The ExecuTorch dependency can also be added to the package file manually. See [Using ExecuTorch on iOS](/TODO.md) for more information. +To get started with Xcode, go to File > Add Package Dependencies. Paste the URL of the ExecuTorch repo into the search bar and select it. Make sure to change the branch name to the desired ExecuTorch version in format “swiftpm-”, (e.g. “swiftpm-0.5.0”). The ExecuTorch dependency can also be added to the package file manually. See [Using ExecuTorch on iOS](using-executorch-ios.md) for more information. #### Runtime APIs -Models can be loaded and run from Swift as follows: -```swift -// TODO Code sample -``` +Models can be loaded and run from Objective-C using the C++ APIs. -For more information on iOS integration, including an API reference, logging setup, and building from source, see [Using ExecuTorch on iOS](/TODO.md). +For more information on iOS integration, including an API reference, logging setup, and building from source, see [Using ExecuTorch on iOS](using-executorch-ios.md). ### C++ ExecuTorch provides C++ APIs, which can be used to target embedded or mobile devices. The C++ APIs provide a greater level of control compared to other language bindings, allowing for advanced memory management, data loading, and platform integration. #### Installation -CMake is the preferred build system for the ExecuTorch C++ runtime. To use with CMake, clone the ExecuTorch repository as a subdirectory of your project, and use CMake's `add_subdirectory("executorch")` to include the dependency. The `executorch` target, as well as kernel and backend targets will be made available to link against. The runtime can also be built standalone to support diverse toolchains. See [Using ExecuTorch with C++](/TODO.md) for a detailed description of build integration, targets, and cross compilation. +CMake is the preferred build system for the ExecuTorch C++ runtime. To use with CMake, clone the ExecuTorch repository as a subdirectory of your project, and use CMake's `add_subdirectory("executorch")` to include the dependency. The `executorch` target, as well as kernel and backend targets will be made available to link against. The runtime can also be built standalone to support diverse toolchains. See [Using ExecuTorch with C++](using-executorch-cpp.md) for a detailed description of build integration, targets, and cross compilation. ``` git clone -b release/0.5 https://github.com/pytorch/executorch.git @@ -199,9 +199,9 @@ For more information on the C++ APIs, see [Running an ExecuTorch Model Using the ExecuTorch provides a high-degree of customizability to support diverse hardware targets. Depending on your use cases, consider exploring one or more of the following pages: - [Export and Lowering](using-executorch-export.md) for advanced model conversion options. -- [Delegates](/TODO.md) for available backends and configuration options. -- [Using ExecuTorch on Android](/TODO.md) and [Using ExecuTorch on iOS](TODO.md) for mobile runtime integration. -- [Using ExecuTorch with C++](/TODO.md) for embedded and mobile native development. -- [Troubleshooting, Profiling, and Optimization](/TODO.md) for developer tooling and debugging. -- [API Reference](/TODO.md) for a full description of available APIs. +- [Backend Overview](backends-overview.md) for available backends and configuration options. +- [Using ExecuTorch on Android](using-executorch-android.md) and [Using ExecuTorch on iOS](using-executorch-ios.md) for mobile runtime integration. +- [Using ExecuTorch with C++](using-executorch-cpp.md) for embedded and mobile native development. +- [Profiling and Debugging](using-executorch-troubleshooting.md) for developer tooling and debugging. +- [API Reference](export-to-executorch-api-reference.md) for a full description of available APIs. - [Examples](https://github.com/pytorch/executorch/tree/main/examples) for demo apps and example code. \ No newline at end of file From 43d3717268e4b6ad5742f46a42a465092f066948 Mon Sep 17 00:00:00 2001 From: Gregory Comer Date: Fri, 21 Feb 2025 00:53:42 -0800 Subject: [PATCH 2/7] Move Android pre-built AAR info into top-level Android page --- docs/source/android-prebuilt-library.md | 37 ------------------------- docs/source/index.rst | 1 + docs/source/using-executorch-android.md | 36 +++++++++++++++++++++++- 3 files changed, 36 insertions(+), 38 deletions(-) delete mode 100644 docs/source/android-prebuilt-library.md diff --git a/docs/source/android-prebuilt-library.md b/docs/source/android-prebuilt-library.md deleted file mode 100644 index 324c63376c5..00000000000 --- a/docs/source/android-prebuilt-library.md +++ /dev/null @@ -1,37 +0,0 @@ -# Using Android prebuilt libraries (AAR) - -We provide two prebuilt Android libraries (AAR), `executorch.aar` for generic use case (image/audio processing) and `executorch_llama.aar` for LLAMA use case. - -## Contents of libraries -- `executorch.aar` - - [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/src/main/java/org/pytorch/executorch) - - JNI contains the JNI binding for [NativePeer.java](https://github.com/pytorch/executorch/blob/main/extension/android/src/main/java/org/pytorch/executorch/NativePeer.java) and ExecuTorch native library, including core ExecuTorch runtime libraries, XNNPACK backend, Portable kernels, Optimized kernels, and Quantized kernels. - - Comes with two ABI variants, arm64-v8a and x86_64. -- `executorch_llama.aar` - - [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/src/main/java/org/pytorch/executorch) (Note: it contains the same Java classes as the previous Java, but it does not contain the JNI binding for generic Module/NativePeer Java code). - - JNI contains the JNI binding for [LlamaModule.java](https://github.com/pytorch/executorch/blob/main/extension/android/src/main/java/org/pytorch/executorch/LlamaModule.java) and ExecuTorch native library, including core ExecuTorch runtime libraries, XNNPACK backend, Portable kernels, Optimized kernels, Quantized kernels, and LLAMA-specific Custom ops library. - - Comes with two ABI variants, arm64-v8a and x86_64. - -## Downloading AAR -[executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar) -[executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar.sha256sums) - -## Using prebuilt libraries - -To add the Java library to your app, simply download the AAR, and add it to your gradle build rule. - -In your app working directory, such as example executorch/examples/demo-apps/android/LlamaDemo, -``` -mkdir -p app/libs -curl https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar -o app/libs/executorch.aar -``` - -And include it in gradle: -``` -# app/build.grardle.kts -dependencies { - implementation(files("libs/executorch.aar")) -} -``` - -Now you can compile your app with the ExecuTorch Android library. diff --git a/docs/source/index.rst b/docs/source/index.rst index 5061d8b056d..a7e538d700f 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -100,6 +100,7 @@ Topics in this section will help you get started with ExecuTorch. :caption: Backends :hidden: + backends-overview backends-xnnpack backends-coreml backends-mps diff --git a/docs/source/using-executorch-android.md b/docs/source/using-executorch-android.md index cedb355810b..063e83170d0 100644 --- a/docs/source/using-executorch-android.md +++ b/docs/source/using-executorch-android.md @@ -4,7 +4,41 @@ To use from Android, ExecuTorch provides Java API bindings and Android platform ## Installation -TODO Instructions on downloading the pre-built AAR. Replace with Maven/Gradle package management when available. +We provide two prebuilt Android libraries (AAR), `executorch.aar` for generic use case (image/audio processing) and `executorch_llama.aar` for LLAMA use case. + +## Contents of libraries +- `executorch.aar` + - [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/src/main/java/org/pytorch/executorch) + - JNI contains the JNI binding for [NativePeer.java](https://github.com/pytorch/executorch/blob/main/extension/android/src/main/java/org/pytorch/executorch/NativePeer.java) and ExecuTorch native library, including core ExecuTorch runtime libraries, XNNPACK backend, Portable kernels, Optimized kernels, and Quantized kernels. + - Comes with two ABI variants, arm64-v8a and x86_64. +- `executorch_llama.aar` + - [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/src/main/java/org/pytorch/executorch) (Note: it contains the same Java classes as the previous Java, but it does not contain the JNI binding for generic Module/NativePeer Java code). + - JNI contains the JNI binding for [LlamaModule.java](https://github.com/pytorch/executorch/blob/main/extension/android/src/main/java/org/pytorch/executorch/LlamaModule.java) and ExecuTorch native library, including core ExecuTorch runtime libraries, XNNPACK backend, Portable kernels, Optimized kernels, Quantized kernels, and LLAMA-specific Custom ops library. + - Comes with two ABI variants, arm64-v8a and x86_64. + +## Downloading AAR +[executorch.aar](https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar) +[executorch.aar.sha256sums](https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar.sha256sums) + +## Using prebuilt libraries + +To add the Java library to your app, simply download the AAR, and add it to your gradle build rule. + +In your app working directory, such as example executorch/examples/demo-apps/android/LlamaDemo, +``` +mkdir -p app/libs +curl https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar -o app/libs/executorch.aar +``` + +And include it in gradle: +``` +# app/build.grardle.kts +dependencies { + implementation(files("libs/executorch.aar")) +} +``` + +Now you can compile your app with the ExecuTorch Android library. ### Building from Source From e62bc678151f4ff08fa873951b67ca8b154760ef Mon Sep 17 00:00:00 2001 From: Gregory Comer Date: Fri, 21 Feb 2025 01:21:29 -0800 Subject: [PATCH 3/7] Add placeholder backend overview --- docs/source/backends-overview.md | 21 ++++++++++++++++++++- docs/source/using-executorch-export.md | 16 ++++++++-------- 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/docs/source/backends-overview.md b/docs/source/backends-overview.md index 7b86b8bae7c..20a5b9f684a 100644 --- a/docs/source/backends-overview.md +++ b/docs/source/backends-overview.md @@ -1 +1,20 @@ -# Backend Overview \ No newline at end of file +# Backend Overview + +ExecuTorch backends provide hardware acceleration for a specific hardware target. In order to achieve maximum performance on target hardware, ExecuTorch optimizes the model for a specific backend during the export and lowering process. This means that the resulting .pte file is specialized for the specific hardware. In order to deploy to multiple backends, such as Core ML on iOS and Arm CPU on Android, it is common to generate a dedicated .pte file for each. + +The choice of hardware backend is informed by the hardware that the model is intended to be deployed on. Each backend has specific hardware requires and level of model support. See the documentation for each hardware backend for more details. + +As part of the .pte file creation process, ExecuTorch identifies portions of the model (partitions) that are supported for the given backend. These sections are processed by the backend ahead of time to support efficient execution. Portions of the model that are not supported on the delegate, if any, are executed using the portable fallback implementation on CPU. This allows for partial model acceleration when not all model operators are supported on the backend, but may have negative performance implications. In addition, multiple partitioners can be specified in order of priority. This allows for operators not supported on GPU to run on CPU via XNNPACK, for example. + +### Available Backends + +Commonly used hardware backends are listed below. For mobile, consider using XNNPACK for Android and XNNPACK or Core ML for iOS. To create a .pte file for a specific backend, pass the appropriate partitioner class to `to_edge_transform_and_lower`. See the appropriate backend documentation and the [Export and Lowering](#export-and-lowering) section below for more information. + +- [XNNPACK (Mobile CPU)](backends-xnnpack.md) +- [Core ML (iOS)](backends-coreml.md) +- [Metal Performance Shaders (iOS GPU)](backends-mps.md) +- [Vulkan (Android GPU)](backends-vulkan.md) +- [Qualcomm NPU](backends-qualcomm.md) +- [MediaTek NPU](backends-mediatek.md) +- [Arm Ethos-U NPU](backends-arm-ethos-u.md) +- [Cadence DSP](backends-cadence.md) \ No newline at end of file diff --git a/docs/source/using-executorch-export.md b/docs/source/using-executorch-export.md index 7357ca4554d..62a52edf839 100644 --- a/docs/source/using-executorch-export.md +++ b/docs/source/using-executorch-export.md @@ -32,14 +32,14 @@ As part of the .pte file creation process, ExecuTorch identifies portions of the Commonly used hardware backends are listed below. For mobile, consider using XNNPACK for Android and XNNPACK or Core ML for iOS. To create a .pte file for a specific backend, pass the appropriate partitioner class to `to_edge_transform_and_lower`. See the appropriate backend documentation and the [Export and Lowering](#export-and-lowering) section below for more information. -- [XNNPACK (Mobile CPU)](native-delegates-executorch-xnnpack-delegate.md) -- [Core ML (iOS)](native-delegates-executorch-coreml-delegate.md) -- [Metal Performance Shaders (iOS GPU)](native-delegates-executorch-mps-delegate.md) -- [Vulkan (Android GPU)](native-delegates-executorch-vulkan-delegate.md) -- [Qualcomm NPU](native-delegates-executorch-qualcomm-delegate.md) -- [MediaTek NPU](native-delegates-executorch-mediatek-delegate.md) -- [Arm Ethos-U NPU](native-delegates-executorch-arm-ethos-u-delegate.md) -- [Cadence DSP](native-delegates-executorch-cadence-delegate.md) +- [XNNPACK (Mobile CPU)](backends-xnnpack.md) +- [Core ML (iOS)](backends-coreml.md) +- [Metal Performance Shaders (iOS GPU)](backends-mps.md) +- [Vulkan (Android GPU)](backends-vulkan.md) +- [Qualcomm NPU](backends-qualcomm.md) +- [MediaTek NPU](backends-mediatek.md) +- [Arm Ethos-U NPU](backends-arm-ethos-u.md) +- [Cadence DSP](backends-cadence.md) ## Model Preparation From 8752431ef9f213ed86f71f3bb54d685c4ac8614b Mon Sep 17 00:00:00 2001 From: Gregory Comer Date: Fri, 21 Feb 2025 02:01:35 -0800 Subject: [PATCH 4/7] Add placeholder troubleshooting docs --- docs/source/using-executorch-ios.md | 2 +- .../using-executorch-troubleshooting.md | 19 ++++++++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/source/using-executorch-ios.md b/docs/source/using-executorch-ios.md index 68599065f6a..029914eb498 100644 --- a/docs/source/using-executorch-ios.md +++ b/docs/source/using-executorch-ios.md @@ -106,7 +106,7 @@ git clone https://github.com/pytorch/executorch.git --depth 1 --recurse-submodul python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip ``` -4. Install the required dependencies, including those needed for the backends like [Core ML](build-run-coreml.md) or [MPS](build-run-mps.md), if you plan to build them as well: +4. Install the required dependencies, including those needed for the backends like [Core ML](backends-coreml.md) or [MPS](backends-mps.md), if you plan to build them as well: ```bash ./install_executorch.sh --pybind coreml mps xnnpack diff --git a/docs/source/using-executorch-troubleshooting.md b/docs/source/using-executorch-troubleshooting.md index 406c1ed8347..60d95ac4d97 100644 --- a/docs/source/using-executorch-troubleshooting.md +++ b/docs/source/using-executorch-troubleshooting.md @@ -1,3 +1,20 @@ # Profiling and Debugging -Placeholder for top-level troubleshooting, profiling, and devtool docs \ No newline at end of file +ExecuTorch + +## General Troubleshooting Steps + +- To troubleshoot failure of runtime API calls, such as loading or running a model, ensure that ExecuTorch framework logging is enabled. See [Logging](using-executorch-runtime-integration.md#logging) for more information. +- As a prelimatinary step to troubleshoot slow run times, ensure that performance testing is being done in a release build, and that the model is delegated. See [Inference is Slow](using-executorch-faqs.md#inference-is-slow--performance-troubleshooting) for more information. +- Check [Frequently Asked Questions](using-executorch-faqs.md) for common issues and questions encountered during install, model export, and runtime integration. + +## Developer Tools + +The ExecuTorch developer tools, or devtools, are a collection of tooling for troubleshooting model performance, numerics, and resource utilization. See [Introduction to the ExecuTorch Developer Tools](devtools-overview.md) for an overview of the available developer tools and usage. + +## Next Steps + +- [Frequently Asked Questions](using-executorch-faqs.md) for solutions to commonly encountered questions and issues. +- [Introduction to the ExecuTorch Developer Tools](runtime-profiling.md) for a high-level introduction to available developer tooling. +- [Using the ExecuTorch Developer Tools to Profile a Model](tutorials/devtools-integration-tutorial.md) for information on runtime performance profiling. +- [Inspector APIs](runtime-profiling.md) for reference material on trace inspector APIs. \ No newline at end of file From e450f8402d1e7e5062c8f927b37185bc99b19178 Mon Sep 17 00:00:00 2001 From: Gregory Comer Date: Fri, 21 Feb 2025 04:04:22 -0800 Subject: [PATCH 5/7] Populate top-level C++ API doc --- docs/source/using-executorch-cpp.md | 43 ++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/source/using-executorch-cpp.md b/docs/source/using-executorch-cpp.md index 12728a6f575..e2eaaac7e94 100644 --- a/docs/source/using-executorch-cpp.md +++ b/docs/source/using-executorch-cpp.md @@ -1,3 +1,44 @@ # Using ExecuTorch with C++ -Placeholder for top-level C++ documentation \ No newline at end of file +In order to support a wide variety of devices, from high-end mobile phones down to tiny embedded systems, ExecuTorch provides an API surface with a high degree of customizability. The C++ APIs expose advanced configuration options, such as controlling memory allocation, placement, and data loading. To meet the needs of both application and embedded programming, ExecuTorch provides a low-level, highly-customizable core set of APIs, and set of high-level extensions, which abstract away many of the low-level details that are not relevant for mobile application programming. + +## High-Level APIs + +The C++ `Module` class provides the high-level interface to load and execute a model from C++. It is responsible for loading the .pte file, configuring memory allocation and placement, and running the model. The Module constructor takes a file path and provides a simplified `forward()` method to run the model. + +In addition the Module class, the tensor extension provides an encapsulated interface to define and manage tensor memory. It provides the `TensorPtr` class, which is a "fat" smart pointer. It provides ownership over the tensor data and metadata, such as size and strides. The `make_tensor_ptr` and `from_blob` methods, defined in `tensor.h`, provide owning and non-owning tensor creation APIs, respectively. + +```cpp +#include +#include + +using namespace ::executorch::extension; + +// Load the model. +Module module("/path/to/model.pte"); + +// Create an input tensor. +float input[1 * 3 * 256 * 256]; +auto tensor = from_blob(input, {1, 3, 256, 256}); + +// Perform an inference. +const auto result = module.forward(tensor); + +if (result.ok()) { + // Retrieve the output data. + const auto output = result->at(0).toTensor().const_data_ptr(); +} +``` + +For more information on the Module class, see [Running an ExecuTorch Model Using the Module Extension in C++](extension-module.md). For information on high-level tensor APIs, see [Managing Tensor Memory in C++](extension-tensor.md). + +## Low-Level APIs + +Running a model using the low-level runtime APIs allows for a high-degree of control over memory allocation, placement, and loading. This allows for advanced use cases, such as placing allocations in specific memory banks or loading a model without a file system. For an end to end example using the low-level runtime APIs, see [Running an ExecuTorch Model in C++ Tutorial](running-a-model-cpp-tutorial.md). + +## Next Steps + +- [Runtime API Reference](executorch-runtime-api-reference.md) for documentation on the available C++ runtime APIs. +- [Running an ExecuTorch Model Using the Module Extension in C++](extension-module.md) for information on the high-level Module API. +- [Managing Tensor Memory in C++](extension-tensor.md) for information on high-level tensor APIs. +- [Running an ExecuTorch Model in C++ Tutorial](running-a-model-cpp-tutorial.md) for information on the low-level runtime APIs. \ No newline at end of file From 7d72c5ac54b1be16954ccf7f818e30dd32c9c373 Mon Sep 17 00:00:00 2001 From: Gregory James Comer Date: Fri, 21 Feb 2025 15:42:15 -0800 Subject: [PATCH 6/7] Clean up additional doc placeholders and fix broken links --- docs/source/backends-arm-ethos-u.md | 6 ++--- docs/source/backends-cadence.md | 6 ++--- docs/source/backends-coreml.md | 6 ++--- docs/source/backends-mediatek.md | 8 +++---- docs/source/backends-mps.md | 6 ++--- docs/source/backends-overview.md | 4 ++-- docs/source/backends-qualcomm.md | 6 ++--- docs/source/getting-started.md | 16 ++++++++------ docs/source/index.rst | 5 ++++- ...lama3-qualcomm-ai-engine-direct-backend.md | 2 +- docs/source/llm/getting-started.md | 4 ++-- docs/source/runtime-overview.md | 2 +- .../tutorial-xnnpack-delegate-lowering.md | 2 +- .../using-executorch-building-from-source.md | 6 ++--- docs/source/using-executorch-cpp.md | 22 ++++++++++++++++++- .../using-executorch-runtime-integration.md | 4 ++-- .../using-executorch-troubleshooting.md | 4 ++-- 17 files changed, 67 insertions(+), 42 deletions(-) diff --git a/docs/source/backends-arm-ethos-u.md b/docs/source/backends-arm-ethos-u.md index d3c50b5bc78..a9f89bdabc8 100644 --- a/docs/source/backends-arm-ethos-u.md +++ b/docs/source/backends-arm-ethos-u.md @@ -7,8 +7,8 @@ :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites * [Introduction to ExecuTorch](./intro-how-it-works.md) -* [Setting up ExecuTorch](./getting-started-setup.md) -* [Building ExecuTorch with CMake](./runtime-build-and-cross-compilation.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) ::: :::{grid-item-card} What you will learn in this tutorial: @@ -280,7 +280,7 @@ The `generate_pte_file` function in `run.sh` script produces the `.pte` files ba ExecuTorch's CMake build system produces a set of build pieces which are critical for us to include and run the ExecuTorch runtime with-in the bare-metal environment we have for Corstone FVPs from Ethos-U SDK. -[This](./runtime-build-and-cross-compilation.md) document provides a detailed overview of each individual build piece. For running either variant of the `.pte` file, we will need a core set of libraries. Here is a list, +[This](./using-executorch-building-from-source.md) document provides a detailed overview of each individual build piece. For running either variant of the `.pte` file, we will need a core set of libraries. Here is a list, - `libexecutorch.a` - `libportable_kernels.a` diff --git a/docs/source/backends-cadence.md b/docs/source/backends-cadence.md index a07e4b61598..278a2f9ec2b 100644 --- a/docs/source/backends-cadence.md +++ b/docs/source/backends-cadence.md @@ -17,9 +17,9 @@ On top of being able to run on the Xtensa HiFi4 DSP, another goal of this tutori ::: :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites -* [Introduction to ExecuTorch](intro-how-it-works.md) -* [Setting up ExecuTorch](getting-started-setup.md) -* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +* [Introduction to ExecuTorch](./intro-how-it-works.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) ::: :::: diff --git a/docs/source/backends-coreml.md b/docs/source/backends-coreml.md index e46531fa680..804f3fe3a93 100644 --- a/docs/source/backends-coreml.md +++ b/docs/source/backends-coreml.md @@ -11,9 +11,9 @@ Core ML delegate uses Core ML APIs to enable running neural networks via Apple's ::: :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites -* [Introduction to ExecuTorch](intro-how-it-works.md) -* [Setting up ExecuTorch](getting-started-setup.md) -* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +* [Introduction to ExecuTorch](./intro-how-it-works.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) * [ExecuTorch iOS Demo App](demo-apps-ios.md) ::: :::: diff --git a/docs/source/backends-mediatek.md b/docs/source/backends-mediatek.md index 0ce1d89e785..5bf99553b7d 100644 --- a/docs/source/backends-mediatek.md +++ b/docs/source/backends-mediatek.md @@ -11,9 +11,9 @@ MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices ::: :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites -* [Introduction to ExecuTorch](intro-how-it-works.md) -* [Setting up ExecuTorch](getting-started-setup.md) -* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +* [Introduction to ExecuTorch](./intro-how-it-works.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) ::: :::: @@ -91,4 +91,4 @@ cd executorch ```bash export LD_LIBRARY_PATH=::$LD_LIBRARY_PATH - ``` \ No newline at end of file + ``` diff --git a/docs/source/backends-mps.md b/docs/source/backends-mps.md index 0da22ad579f..4947e3e0aea 100644 --- a/docs/source/backends-mps.md +++ b/docs/source/backends-mps.md @@ -12,9 +12,9 @@ The MPS backend device maps machine learning computational graphs and primitives ::: :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites -* [Introduction to ExecuTorch](intro-how-it-works.md) -* [Setting up ExecuTorch](getting-started-setup.md) -* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +* [Introduction to ExecuTorch](./intro-how-it-works.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) * [ExecuTorch iOS Demo App](demo-apps-ios.md) * [ExecuTorch iOS LLaMA Demo App](llm/llama-demo-ios.md) ::: diff --git a/docs/source/backends-overview.md b/docs/source/backends-overview.md index 20a5b9f684a..dd3aa0354bc 100644 --- a/docs/source/backends-overview.md +++ b/docs/source/backends-overview.md @@ -8,7 +8,7 @@ As part of the .pte file creation process, ExecuTorch identifies portions of the ### Available Backends -Commonly used hardware backends are listed below. For mobile, consider using XNNPACK for Android and XNNPACK or Core ML for iOS. To create a .pte file for a specific backend, pass the appropriate partitioner class to `to_edge_transform_and_lower`. See the appropriate backend documentation and the [Export and Lowering](#export-and-lowering) section below for more information. +Commonly used hardware backends are listed below. For mobile, consider using XNNPACK for Android and XNNPACK or Core ML for iOS. To create a .pte file for a specific backend, pass the appropriate partitioner class to `to_edge_transform_and_lower`. See the appropriate backend documentation for more information. - [XNNPACK (Mobile CPU)](backends-xnnpack.md) - [Core ML (iOS)](backends-coreml.md) @@ -17,4 +17,4 @@ Commonly used hardware backends are listed below. For mobile, consider using XNN - [Qualcomm NPU](backends-qualcomm.md) - [MediaTek NPU](backends-mediatek.md) - [Arm Ethos-U NPU](backends-arm-ethos-u.md) -- [Cadence DSP](backends-cadence.md) \ No newline at end of file +- [Cadence DSP](backends-cadence.md) diff --git a/docs/source/backends-qualcomm.md b/docs/source/backends-qualcomm.md index 9001ae716c6..041c3d7fdc3 100644 --- a/docs/source/backends-qualcomm.md +++ b/docs/source/backends-qualcomm.md @@ -14,9 +14,9 @@ Qualcomm AI Engine Direct is also referred to as QNN in the source and documenta ::: :::{grid-item-card} Tutorials we recommend you complete before this: :class-card: card-prerequisites -* [Introduction to ExecuTorch](intro-how-it-works.md) -* [Setting up ExecuTorch](getting-started-setup.md) -* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) +* [Introduction to ExecuTorch](./intro-how-it-works.md) +* [Getting Started](./getting-started.md) +* [Building ExecuTorch with CMake](./using-executorch-building-from-source.md) ::: :::: diff --git a/docs/source/getting-started.md b/docs/source/getting-started.md index 24e37b5e9da..47b8921b832 100644 --- a/docs/source/getting-started.md +++ b/docs/source/getting-started.md @@ -29,7 +29,9 @@ The following are required to install the ExecuTorch host libraries, needed to e
## Preparing the Model -Exporting is the process of taking a PyTorch model and converting it to the .pte file format used by the ExecuTorch runtime. This is done using Python APIs. PTE files for common models, such as Llama 3, can be found on HuggingFace under [ExecuTorch Community](https://huggingface.co/executorch-community). These models have been exported and lowered for ExecuTorch, and can be directly deployed without needing to go through the lowering process. +Exporting is the process of taking a PyTorch model and converting it to the .pte file format used by the ExecuTorch runtime. This is done using Python APIs. PTE files for common models, such as Llama 3.2, can be found on HuggingFace under [ExecuTorch Community](https://huggingface.co/executorch-community). These models have been exported and lowered for ExecuTorch, and can be directly deployed without needing to go through the lowering process. + +A complete example of exporting, lowering, and verifying MobileNet V2 is available as a [Colab notebook](https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing). ### Requirements - A PyTorch model. @@ -39,7 +41,7 @@ Exporting is the process of taking a PyTorch model and converting it to the .pte ### Selecting a Backend ExecuTorch provides hardware acceleration for a wide variety of hardware. The most commonly used backends are XNNPACK, for Arm and x86 CPU, Core ML (for iOS), Vulkan (for Android GPUs), and Qualcomm (for Qualcomm-powered Android phones). -For mobile use cases, consider using XNNPACK for Android and Core ML or XNNPACK for iOS as a first step. See [Hardware Backends](using-executorch-export.md#hardware-backends) for more information. +For mobile use cases, consider using XNNPACK for Android and Core ML or XNNPACK for iOS as a first step. See [Hardware Backends](backends-overview.md) for more information. ### Exporting Exporting is done using Python APIs. ExecuTorch provides a high degree of customization during the export process, but the typical flow is as follows: @@ -50,13 +52,13 @@ model = MyModel() # The PyTorch model to export example_inputs = (torch.randn(1,3,64,64),) # A tuple of inputs et_program = - executorch.exir.to_edge_transform_and_lower( - torch.export.export(model, example_inputs) + executorch.exir.to_edge_transform_and_lower( + torch.export.export(model, example_inputs) partitioner=[XnnpackPartitioner()] ).to_executorch() with open(“model.pte”, “wb”) as f: - f.write(et_program.buffer) + f.write(et_program.buffer) ``` If the model requires varying input sizes, you will need to specify the varying dimensions and bounds as part of the `export` call. See [Model Export and Lowering](using-executorch-export.md) for more information. @@ -119,7 +121,7 @@ import org.pytorch.executorch.Tensor; // … -Module model = Module.load(“/path/to/model.pte”) +Module model = Module.load(“/path/to/model.pte”); Tensor input_tensor = Tensor.fromBlob(float_data, new long[] { 1, 3, height, width }); EValue input_evalue = EValue.from(input_tensor); @@ -204,4 +206,4 @@ ExecuTorch provides a high-degree of customizability to support diverse hardware - [Using ExecuTorch with C++](using-executorch-cpp.md) for embedded and mobile native development. - [Profiling and Debugging](using-executorch-troubleshooting.md) for developer tooling and debugging. - [API Reference](export-to-executorch-api-reference.md) for a full description of available APIs. -- [Examples](https://github.com/pytorch/executorch/tree/main/examples) for demo apps and example code. \ No newline at end of file +- [Examples](https://github.com/pytorch/executorch/tree/main/examples) for demo apps and example code. diff --git a/docs/source/index.rst b/docs/source/index.rst index a7e538d700f..1af4e41f2f1 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -91,8 +91,8 @@ Topics in this section will help you get started with ExecuTorch. using-executorch-cpp using-executorch-runtime-integration using-executorch-troubleshooting - using-executorch-faqs using-executorch-building-from-source + using-executorch-faqs .. toctree:: :glob: @@ -140,6 +140,9 @@ Topics in this section will help you get started with ExecuTorch. :hidden: runtime-overview + extension-module + extension-tensor + running-a-model-cpp-tutorial runtime-backend-delegate-implementation-and-linking runtime-platform-abstraction-layer portable-cpp-programming diff --git a/docs/source/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md b/docs/source/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md index 7ed768baf23..c02701a839c 100644 --- a/docs/source/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md +++ b/docs/source/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md @@ -5,7 +5,7 @@ This tutorial demonstrates how to export Llama 3 8B Instruct for Qualcomm AI Eng ## Prerequisites - Set up your ExecuTorch repo and environment if you haven’t done so by following [the Setting up ExecuTorch](../getting-started-setup.md) to set up the repo and dev environment. -- Read [the Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend page](../build-run-qualcomm-ai-engine-direct-backend.md) to understand how to export and run a model with Qualcomm AI Engine Direct Backend on Qualcomm device. +- Read [the Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend page](../backends-qualcomm.md) to understand how to export and run a model with Qualcomm AI Engine Direct Backend on Qualcomm device. - Follow [the README for executorch llama](https://github.com/pytorch/executorch/tree/main/examples/models/llama) to know how to run a llama model on mobile via ExecuTorch. - A Qualcomm device with 16GB RAM - We are continuing to optimize our memory usage to ensure compatibility with lower memory devices. diff --git a/docs/source/llm/getting-started.md b/docs/source/llm/getting-started.md index fd5dd3ba3a2..3bcba08ddfb 100644 --- a/docs/source/llm/getting-started.md +++ b/docs/source/llm/getting-started.md @@ -592,8 +592,8 @@ I'm not sure if you've heard of the "Curse of the Dragon" or not, but it's a ver The delegated model should be noticeably faster compared to the non-delegated model. For more information regarding backend delegateion, see the ExecuTorch guides -for the [XNNPACK Backend](../tutorial-xnnpack-delegate-lowering.md), [Core ML -Backend](../build-run-coreml.md) and [Qualcomm AI Engine Direct Backend](build-run-llama3-qualcomm-ai-engine-direct-backend.md). +for the [XNNPACK Backend](../backends-xnnpack.md), [Core ML +Backend](../backends-coreml.md) and [Qualcomm AI Engine Direct Backend](build-run-llama3-qualcomm-ai-engine-direct-backend.md). ## Quantization diff --git a/docs/source/runtime-overview.md b/docs/source/runtime-overview.md index 1a421fdcc0a..911d0c142e8 100644 --- a/docs/source/runtime-overview.md +++ b/docs/source/runtime-overview.md @@ -157,7 +157,7 @@ For more details about the ExecuTorch runtime, please see: * [Detailed Runtime APIs Tutorial](running-a-model-cpp-tutorial.md) * [Simplified Runtime APIs Tutorial](extension-module.md) -* [Runtime Build and Cross Compilation](runtime-build-and-cross-compilation.md) +* [Building from Source](using-executorch-building-from-source.md) * [Runtime Platform Abstraction Layer](runtime-platform-abstraction-layer.md) * [Runtime Profiling](runtime-profiling.md) * [Backends and Delegates](compiler-delegate-and-partitioner.md) diff --git a/docs/source/tutorial-xnnpack-delegate-lowering.md b/docs/source/tutorial-xnnpack-delegate-lowering.md index d1148511c5f..a469edebd54 100644 --- a/docs/source/tutorial-xnnpack-delegate-lowering.md +++ b/docs/source/tutorial-xnnpack-delegate-lowering.md @@ -12,7 +12,7 @@ In this tutorial, you will learn how to export an XNNPACK lowered Model and run :class-card: card-prerequisites * [Setting up ExecuTorch](./getting-started-setup.md) * [Model Lowering Tutorial](./tutorials/export-to-executorch-tutorial) -* [ExecuTorch XNNPACK Delegate](./native-delegates-executorch-xnnpack-delegate.md) +* [ExecuTorch XNNPACK Delegate](./backends-xnnpack.md) ::: :::: diff --git a/docs/source/using-executorch-building-from-source.md b/docs/source/using-executorch-building-from-source.md index 01ec05b1156..8932c0841bc 100644 --- a/docs/source/using-executorch-building-from-source.md +++ b/docs/source/using-executorch-building-from-source.md @@ -159,7 +159,7 @@ xcode-select --install ``` Run the above command with `--help` flag to learn more on how to build additional backends -(like [Core ML](build-run-coreml.md), [MPS](build-run-mps.md) or XNNPACK), etc. +(like [Core ML](backends-coreml.md), [MPS](backends-mps.md) or XNNPACK), etc. Note, some backends may require additional dependencies and certain versions of Xcode and iOS. 3. Copy over the generated `.xcframework` bundles to your Xcode project, link them against @@ -172,6 +172,6 @@ Check out the [iOS Demo App](demo-apps-ios.md) tutorial for more info. You have successfully cross-compiled `executor_runner` binary to iOS and Android platforms. You can start exploring advanced features and capabilities. Here is a list of sections you might want to read next: -* [Selective build](./kernel-library-selective_build) to build the runtime that links to only kernels used by the program, which can provide significant binary size savings. +* [Selective build](kernel-library-selective-build.md) to build the runtime that links to only kernels used by the program, which can provide significant binary size savings. * Tutorials on building [Android](./demo-apps-android.md) and [iOS](./demo-apps-ios.md) demo apps. -* Tutorials on deploying applications to embedded devices such as [ARM Cortex-M/Ethos-U](./executorch-arm-delegate-tutorial.md) and [XTensa HiFi DSP](./build-run-xtensa.md). +* Tutorials on deploying applications to embedded devices such as [ARM Cortex-M/Ethos-U](backends-arm-ethos-u.md) and [XTensa HiFi DSP](./backends-cadence.md). diff --git a/docs/source/using-executorch-cpp.md b/docs/source/using-executorch-cpp.md index e2eaaac7e94..d0be0e3bd33 100644 --- a/docs/source/using-executorch-cpp.md +++ b/docs/source/using-executorch-cpp.md @@ -36,9 +36,29 @@ For more information on the Module class, see [Running an ExecuTorch Model Using Running a model using the low-level runtime APIs allows for a high-degree of control over memory allocation, placement, and loading. This allows for advanced use cases, such as placing allocations in specific memory banks or loading a model without a file system. For an end to end example using the low-level runtime APIs, see [Running an ExecuTorch Model in C++ Tutorial](running-a-model-cpp-tutorial.md). +## Building with C++ + +ExecuTorch uses CMake as the primary build system. Inclusion of the module and tensor APIs are controlled by the `EXECUTORCH_BUILD_EXTENSION_MODULE` and `EXECUTORCH_BUILD_EXTENSION_TENSOR` CMake options. As these APIs may not be supported on embedded systems, they are disabled by default when building from source. The low-level API surface is always included. To link, add the `executorch` target as a CMake dependency, along with `executorch_module_static` and `executorch_tensor`, if desired. + +``` +# CMakeLists.txt +add_subdirectory("executorch") +... +target_link_libraries( + my_target + PRIVATE executorch + executorch_module_static + executorch_tensor + optimized_native_cpu_ops_lib + xnnpack_backend) +``` + +See [Building from Source](using-executorch-building-from-source.md) for more information on the CMake build process. + ## Next Steps - [Runtime API Reference](executorch-runtime-api-reference.md) for documentation on the available C++ runtime APIs. - [Running an ExecuTorch Model Using the Module Extension in C++](extension-module.md) for information on the high-level Module API. - [Managing Tensor Memory in C++](extension-tensor.md) for information on high-level tensor APIs. -- [Running an ExecuTorch Model in C++ Tutorial](running-a-model-cpp-tutorial.md) for information on the low-level runtime APIs. \ No newline at end of file +- [Running an ExecuTorch Model in C++ Tutorial](running-a-model-cpp-tutorial.md) for information on the low-level runtime APIs. +- [Building from Source](using-executorch-building-from-source.md) for information on CMake build integration. diff --git a/docs/source/using-executorch-runtime-integration.md b/docs/source/using-executorch-runtime-integration.md index 6534e5855cf..08e071e59ab 100644 --- a/docs/source/using-executorch-runtime-integration.md +++ b/docs/source/using-executorch-runtime-integration.md @@ -10,7 +10,7 @@ Logging is sent to STDOUT and STDERR by default on host platforms, and is redire To configure log level when building from source, specify `EXECUTORCH_ENABLE_LOGGING` as on or off and `EXECUTORCH_LOG_LEVEL` as one of debug, info, error, or fatal. Logging is enabled by default in debug builds and disabled in release. Log level defaults to info. -See [Building from Source](TODO) for more information. +See [Building from Source](using-executorch-building-from-source.md) for more information. ``` cmake -b cmake-out -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_LOG_LEVEL=DEBUG ... @@ -50,4 +50,4 @@ The choice of kernel library is transparent to the user when using mobile pre-bu By default, ExecuTorch ships with all supported operator kernels, allowing it to run any supported model at any precision. This comes with a binary size of several megabytes, which may be undesirable for production use cases or resource constrained systems. To minimize binary size, ExecuTorch provides selective build functionality, in order to include only the operators needed to run specific models. -Note the selective build only applies to the portable and optimized kernel libraries. Delegates do not participate in selective build and can be included or excluded by linking indivually. See [Kernel Library Selective Build](kernel-library-selective-build.md) for more information. \ No newline at end of file +Note the selective build only applies to the portable and optimized kernel libraries. Delegates do not participate in selective build and can be included or excluded by linking indivually. See [Kernel Library Selective Build](kernel-library-selective-build.md) for more information. diff --git a/docs/source/using-executorch-troubleshooting.md b/docs/source/using-executorch-troubleshooting.md index 60d95ac4d97..16006802611 100644 --- a/docs/source/using-executorch-troubleshooting.md +++ b/docs/source/using-executorch-troubleshooting.md @@ -1,6 +1,6 @@ # Profiling and Debugging -ExecuTorch +To faciliate model and runtime integration, ExecuTorch provides tools to profile model resource utilization, numerics, and more. This section describes the available troubleshooting tools and steps to resolve issues when integrating ExecuTorch. ## General Troubleshooting Steps @@ -17,4 +17,4 @@ The ExecuTorch developer tools, or devtools, are a collection of tooling for tro - [Frequently Asked Questions](using-executorch-faqs.md) for solutions to commonly encountered questions and issues. - [Introduction to the ExecuTorch Developer Tools](runtime-profiling.md) for a high-level introduction to available developer tooling. - [Using the ExecuTorch Developer Tools to Profile a Model](tutorials/devtools-integration-tutorial.md) for information on runtime performance profiling. -- [Inspector APIs](runtime-profiling.md) for reference material on trace inspector APIs. \ No newline at end of file +- [Inspector APIs](runtime-profiling.md) for reference material on trace inspector APIs. From 198fecddc42cfad0c0433a082d6eeb3360182464 Mon Sep 17 00:00:00 2001 From: Gregory James Comer Date: Fri, 21 Feb 2025 16:08:11 -0800 Subject: [PATCH 7/7] Add env setup instructions for source build --- .../using-executorch-building-from-source.md | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/docs/source/using-executorch-building-from-source.md b/docs/source/using-executorch-building-from-source.md index 8932c0841bc..145b3ef7285 100644 --- a/docs/source/using-executorch-building-from-source.md +++ b/docs/source/using-executorch-building-from-source.md @@ -4,6 +4,98 @@ ExecuTorch uses [CMake](https://cmake.org/) as the primary build system. Even if you don't use CMake directly, CMake can emit scripts for other format like Make, Ninja or Xcode. For information, see [cmake-generators(7)](https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html). +## System Requirements +### Operating System + +We've tested these instructions on the following systems, although they should +also work in similar environments. + + +Linux (x86_64) +- CentOS 8+ +- Ubuntu 20.04.6 LTS+ +- RHEL 8+ + +macOS (x86_64/M1/M2) +- Big Sur (11.0)+ + +Windows (x86_64) +- Windows Subsystem for Linux (WSL) with any of the Linux options + +### Software +* `conda` or another virtual environment manager + - We recommend `conda` as it provides cross-language + support and integrates smoothly with `pip` (Python's built-in package manager) + - Otherwise, Python's built-in virtual environment manager `python venv` is a good alternative. +* `g++` version 7 or higher, `clang++` version 5 or higher, or another + C++17-compatible toolchain. + +Note that the cross-compilable core runtime code supports a wider range of +toolchains, down to C++17. See the [Runtime Overview](./runtime-overview.md) for +portability details. + +## Environment Setup + +### Create a Virtual Environment + +[Install conda on your machine](https://conda.io/projects/conda/en/latest/user-guide/install/index.html). Then, create a virtual environment to manage our dependencies. + ```bash + # Create and activate a conda environment named "executorch" + conda create -yn executorch python=3.10.0 + conda activate executorch + ``` + +### Clone and install ExecuTorch requirements + + ```bash + # Clone the ExecuTorch repo from GitHub + # 'main' branch is the primary development branch where you see the latest changes. + # 'viable/strict' contains all of the commits on main that pass all of the necessary CI checks. + git clone --branch viable/strict https://github.com/pytorch/executorch.git + cd executorch + + # Update and pull submodules + git submodule sync + git submodule update --init + + # Install ExecuTorch pip package and its dependencies, as well as + # development tools like CMake. + # If developing on a Mac, make sure to install the Xcode Command Line Tools first. + ./install_executorch.sh + ``` + + Use the [`--pybind` flag](https://github.com/pytorch/executorch/blob/main/install_executorch.sh#L26-L29) to install with pybindings and dependencies for other backends. + ```bash + ./install_executorch.sh --pybind + + # Example: pybindings with CoreML *only* + ./install_executorch.sh --pybind coreml + + # Example: pybinds with CoreML *and* XNNPACK + ./install_executorch.sh --pybind coreml xnnpack + ``` + + By default, `./install_executorch.sh` command installs pybindings for XNNPACK. To disable any pybindings altogether: + ```bash + ./install_executorch.sh --pybind off + ``` + +> **_NOTE:_** Cleaning the build system +> +> When fetching a new version of the upstream repo (via `git fetch` or `git +> pull`) it is a good idea to clean the old build artifacts. The build system +> does not currently adapt well to changes in build dependencies. +> +> You should also update and pull the submodules again, in case their versions +> have changed. +> +> ```bash +> # From the root of the executorch repo: +> ./install_executorch.sh --clean +> git submodule sync +> git submodule update --init +> ``` + ## Targets Built by the CMake Build System ExecuTorch's CMake build system covers the pieces of the runtime that are