Build XNNPACK as an ExternalProject

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.

[swolchok]

I believe this is necessary to support EXPORT. (We probably should not
land this unless we have a PR stacked on top of this that successfully
converts the build to EXPORT; sending this out so I can get CI going)

[swolchok]

Stack from ghstack (oldest at bottom):

• ->Build XNNPACK as an ExternalProject #12425

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12425

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 114 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 466f669 with merge base b9bb3c1 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t bfc328039dccf4ac40ed0070eaf6ee6f6570541c295950b033f5799da8cac15a /exec failed with exit code 1
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_11-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_12-cpu (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.11_cpu_x86_64
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_12-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.12_cpu_x86_64
• Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t fadf6aad791b371b85879b516c032120ab608c317910ed6d6470a54bf7d2303e /exec failed with exit code 2
• Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command docker exec -t 017f6af67b473e3b879daa7d289e28e9d5f5f526a2d9afb90d5c2ab62fe406de /exec failed with exit code 2
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t bd3ce59a68384cdba3603116e2a174d2e1c0f38452079ca36c46af51deefcd7c /exec failed with exit code 1
• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t b2fc272eec5136c1846fcc51f388fdbbf94d2d74a26a139b7883b0455502d9cd /exec failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 63b1f5d2ff6c94785d0d12fd20bd8a64c148dcb07232e456d71c3e600c0059e7 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 28cfe31a0715c506730adb56dc5e29540d8ceaffbe584fecf0d5f0432af7d7d7 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8273b1f74a1483071f6b68f8e963f4ed62615b6a26b19f8f4c47e5d5cd88db10 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1cacf9452307ff3b01f643117f3148b87c238317bf582f413a4a3ce31f0a824 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0b88853cf376f81c1f606c380a8eb916be10fbf601c4be19d319854c62c9a365 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3d8e4955ac5d9604c96060964b00709050db56980b4fb8ffb1c6ad5dbf54328e /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 3f094941054b39d389c5ca197d7de9b6dcec71f470ccb573c1182c137a666061 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 84ba37cfb78286195861f86d77d7942b133575af10d0a4930e36851febaa80cc /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 97fc13981f42d32ea996ca1795ac418e1fd5946195387528349a1d5f790a0e11 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t a01188a82af9995c7706878bd0cd300c8f08e82312e4e1fcf74d51b5861258d3 /exec failed with exit code 2
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 0abc71a8451c6675fa130e499f45abe73b7eb80db0671b3a5eb7d0b4d4b97e90 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t f4f6a27cd1edf7051d3f6fdfeac9aa9bd6cd5ba4572b3d4ea42b67beabc53663 /exec failed with exit code 2
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t 6bdd7cf35fd6294a4ac206cc0a8130113bd7381387f74f8c95267a0c2b81209c /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f7af66449d35dc6da1ddad0bf7c4a210b6d59a7978ef342722a07af45fb15364 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t f8a4d20fa83523a5db9b2b49b025bebdc27c1849f6882064e071de4886a51ef3 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6c5ecb58089849a7f4aba0249580c7fab7329db9bf1a8c8a7f2f5b6d94945631 /exec failed with exit code 1
• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6ce89bcdce10da37e47971a37b80bad9b6c4347f3fa05e226f018d4aa7ad5b4f /exec failed with exit code 1
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 6a119f65a8bfeb7e11827f0ffdc321f9d7a63e0a855a59115bb20ecef9abff3d /exec failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t b799036f587c948c9c5aa43e28e41b72f5022cbef6284eda37bf07c9dba31212 /exec failed with exit code 1
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 30bca34364915ccbbc1c6ae62f4fb1de3f35ad25bf5190e793e278595d1d9dca /exec failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 73987e97baa3a062039a636d24ea6d20aafd9ccda08821ddad0adf7fd3ab4900 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 2a16cdf7cf3069a6b9b80f3ffd92047ba1f11fb889409b0ae34198c7953cd1e5 /exec failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a98f8038d2125ccfd699d403ff9e0e01b62eba65ec53014273119ff4321b0c29 /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f74974262c8cf3c0a9735dbcc4d8b1b8e336df48d39048962ea043aa707ed8c /exec failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ab70b0a9e50ed3b65a67ed97295abfae0a090f42ca09acb9bfd2ad7f6c789b3a /exec failed with exit code 1
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t bc11cd713a7cb9008ac48832fe92605cfd960730dd2dec77f80b0267c9258d0e /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 818f1bb1d0e9e38ead16e7bc6eb5e6bad344c97a0d59b51319fe11dc1421770d /exec failed with exit code 1
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t c8c000cc82de941584ba949207f6771801add675ba4608ec99012cb9198e796e /exec failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t a9dee20ba0d80668852e764e7a7691b45e8f0d215f6611473469ef64288dfefa /exec failed with exit code 1
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t eaab8c803e0f0a00bbfde37b37321052fa7f1ecbf5f526c69e65f09bbf6334f6 /exec failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 998e68d10589d2d4208764e4e4837fd0ef9ec2db454d540c5f5bfbb6978a1318 /exec failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 3a0bba71ed92f22e283fac0d737a45e4664d968b86c18c2cbdcebb3807e53ba1 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t e98d64cd7b0a746dfbfa9610990ecf1cab1f511c1d03028ef90dd07634f2a838 /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 24480b215d70e811ecd5f8e1de148a8ee5011a5dab9b020718119e2b39d0de1f /exec failed with exit code 1
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t feca6e3531d99f1e093b3b3e7af3ab9c6a4fbe8ada8412491cf4a3a1f7f033ef /exec failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t f3be2ff97df1002c0f44abc10a1540a34c2b75c00dc36619ea47f2c505a66b9f /exec failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t b25c796ddfb3bb902b774ddab14acf1b35ba1db4ed1bb785d50e6dda1de1a14d /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 936a6a09858fc205e61283819ef0917b52e59be27e5fbffb8e9ebbe6caff878a /exec failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t c7277867a1ae487f9c031f5def2ce24310f7be0ff6a50408bccdbcaf1ead1aaa /exec failed with exit code 1
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ca972fb67ddf3ee4ff046e74c3895a521f8415cbd270f3d2e023562865a3f1d /exec failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t a713b659116cdd33337975268587f46ebf216dadc7e9554c3c9d1b2f308add0f /exec failed with exit code 1
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c3ca2a0781ddbff82d593e468c75e7acf22feb3e8f484d2c789eef7f9998e45 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 57036cccd24540edadf4bdbe8edcbe591acd58c304861b3eb0a01944a4004678 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 5b08a71389d5a13c5bd7c131855a733af3fae8e1d67ff60d92741098b07a2d71 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 91f3c35968a1ff350292de7575b6d43eafaa803b6e17e483b03c3e12154044d1 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 56ede46469a58b7319c80d87a8e8385699eb526b829aed9c5f887623168d1530 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 745aa0b7543282b44833f27ea8c45628b397d3320cfa16b749eab0ccf4726d4e /exec failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 85c9562c1cfda50448a12e424a5ad5c3cfdd8a90bbe9e77d8b6d5748c29106b3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 522051e97a44e902afe84e23ad97aaa7bb4dc83b8c895b114e133b15ed7d82f3 /exec failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t a82f174bed4525f147674fccba8553d6bc14ddc71a656d2dc8bdfecb6cb68c01 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 43612459157e9c547dbfe01993a99b95cd34f7a8eff17c1cbf4fd3651b9e60fc /exec failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c828665dd71ce2095b2d9d2742d3d475981a6517e62183064f29f74ec71ca8a /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 369137d30021a3ef4d276c49b1df9784e4ab200883d20779330d9fc8f36d39d9 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 76c9f6477502edace72bd74e76783ef09fef8aa41167747c859cf00e059b3daa /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 793c5c21bcea52e1801607a5092b37011ef4c00eeb6c1f91543a9f30497975ab /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t beb0096986a422024f2703f3aafd985bbc07d7bdaf8d755a0615df3200f56e5f /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t a9b9c33ed77be0bfcf54e2d61553d7c7a024c3525452f6bed2ea6b8e64d09cc8 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t d0d179cbd1b84a9162136cb638a08efaf2d771f6c403129b50d5015273c9c507 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t d30b2de71572fc56b0aa2884ca5d67891a07988c6180e372cc29442498468e37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6887a59fdd30fbc847fcf791bb05d57fce68b4dce740ae7905fe7ea3de1377 /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 935819f2bda8366428bbedbd80a8f843739603626b5986b2030bd85254a3f203 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 254b840ecb4d40e8d07d407d033623b94a0f866ddd1427de2a233fcac980d3a4 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 66e394feb35c5a51ed17c0b57d99b65821803196c8f65ff65e43c6435779cffa /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
  RuntimeError: Command docker exec -t bd38c6a114cdec4d57ca191838ab3384867ed3e03ef30c703358488564baaa58 /exec failed with exit code 1
• trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
  RuntimeError: Command docker exec -t 4f583ff256f032a2752c6681e040b164f1def621c1facd765c74a93ccd60e55c /exec failed with exit code 1
• trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2595133849e7ba02192363d90d8f5feadea24b52bc2a49358511df056318e4d6 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 97c723e9a3aa10c11829cad689e4b4de72b27f12cc754a6d1802dec83f62d902 /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d6100cc0692c6182f7f12a5297e94b1e280fe9c296cdcb204c2f9da8e88687f /exec failed with exit code 1
• trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a5322ec262cd1b9bea16031999c31e23e4203b3f0a020e5731b23543930b127 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command docker exec -t ff929212e9c034d1277cda6194f74361502079c3c104096b6c9f29e22dfe073b /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command docker exec -t e2d93f3f8c6ccf4c5be7b302f4f0e43ae86db09dcc08e1ad45f345c3f9330a08 /exec failed with exit code 1
• trunk / test-arm-ootb-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 974fcf075b03b18be1177a3129736aaee72cb2c17dff44c46caa7d3bb9b3e491 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
  RuntimeError: Command docker exec -t cfaf9013633cedacab895bcc1d7d91d66c4eb477db2b525de91c126a64733315 /exec failed with exit code 1
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
  RuntimeError: Command docker exec -t 640cd77284f1b0c6d4e5e8ccc6696cf7793478884c65c0c59cbe3ede7cf104ba /exec failed with exit code 1
• trunk / test-huggingface-transformers (allenai/OLMo-1B-hf) / linux-job (gh)
  RuntimeError: Command docker exec -t b18c5e85324ff93aa096fd99fee388b4c9c561607baf5295d71220b3977d06b9 /exec failed with exit code 1
• trunk / test-huggingface-transformers (google/gemma-3-1b-it) / linux-job (gh)
  RuntimeError: Command docker exec -t ce25de3fedd1b7192ab0f2c55bf4fe5d9ef511b6c2742c62620d7cbc4a06ed35 /exec failed with exit code 1
• trunk / test-huggingface-transformers (HuggingFaceTB/SmolLM2-135M) / linux-job (gh)
  RuntimeError: Command docker exec -t c83ed46641510a66f5154a4d50c9e15e2bb1557bf585177aa5d8a5b4baff74c0 /exec failed with exit code 1
• trunk / test-huggingface-transformers (meta-llama/Llama-3.2-1B) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d94aea2b908740ba5614c7a77cf98218f8cd2ec4cc982a49bfb5651539e77e7 /exec failed with exit code 1
• trunk / test-huggingface-transformers (Qwen/Qwen3-0.6B) / linux-job (gh)
  RuntimeError: Command docker exec -t 77be44c500a3a6f8c1f49d8e6de4e171eb2f4febde917d4fa48238a5bf79417b /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 04c65370501a20d7006527f187be4cb65f01bdfd6dd214d70a6892cd630f3ddf /exec failed with exit code 2
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 39d4aefd365733ff4933031554ed9b6824f5387db5b3566b76c9e81fe176c534 /exec failed with exit code 1
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 2ed8d67090895bf1840221129a19c440c4355fbd737e3c60ab461048f9d08b05 /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 771374f9936b95fc8b2ca2a500d602c5ab35da61a16e6585c4160d29a407ee63 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
  RuntimeError: Command docker exec -t 12f50835a766cdb26f55d00d77d732c9c70c31c508a256a8c7bd1b87ac08ae9e /exec failed with exit code 2
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f2760d75b32cbd4f81930a00b5dff5274a8f5a97d49de55f8be8c5f3524eb45 /exec failed with exit code 1
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  RuntimeError: Command docker exec -t 20421f5e1f301f56be9b393e6a6d4009e260603977092879a0b926148ed32953 /exec failed with exit code 2
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 098c9e6a04ee4bd00bbe18b3ad51531cb20c1e667a985f57a70e48a85d3e92af /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t ecda6dc19017d76768a7b5f3d69637a8b655eb281652e7f646f90fb0016eda2c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
  RuntimeError: Command docker exec -t 492411b0963f8a27ff6b4e612bec9b2514043ff5193144386bd3b7fc13bda003 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
  RuntimeError: Command docker exec -t 6926884d881085f324dc628d9160edffb80d25679ef2b3df71ad818f43cf0ffc /exec failed with exit code 1
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
  RuntimeError: Command docker exec -t c4f24eee6ff5a44a909fc7a70dfb88c2e23082acdce72a06c14223565be414bd /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command docker exec -t c5c88a5574619e13639c33e96a2df955d9c7a5c2c451dd42e5b7e3c7ff67b877 /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d834b66ebad9a312a46d238711686417638b0df5a15a4cd14ccf6807429ce3b /exec failed with exit code 1
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
  RuntimeError: Command docker exec -t 958d590f075675040d0f03abff1244f5985bbd7d21785a7a2dd431fc3db2a6be /exec failed with exit code 1
• trunk / test-qnn-model (fp32, vit) / linux-job (gh)
  RuntimeError: Command docker exec -t 3e54f25473d598d3d44e95efffa9d7ffff3e9401d0c195ddea710aa6ab78ff6c /exec failed with exit code 1
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
  RuntimeError: Command docker exec -t 4169567a5eb2b5e6c61bbb8fe3b6d827ccc4597b8280f7aa9ff76fc1110d0fc1 /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
  RuntimeError: Command docker exec -t 6669acee15029908e239019fa79d0ff0144544f6bcc997bdefa23b674b4775aa /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
  RuntimeError: Command docker exec -t 43ed9afdf614194eaa8a2345a8aebeeec9a113ed3994594c2faa914a901ee57d /exec failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
  RuntimeError: Command docker exec -t 18cb16b02cecc8b11acd16ac315e6c0c4ba28a87de9e91921b5fb81b54ed8463 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 09f87b11098f8c78ed919ffc9d36f7c666122a4f377b2bfd552db222bd97412f /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  backends/apple/coreml/test/test_coreml_partitioner.py::TestCoreMLPartitioner::test_ops_to_not_decompose

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

Apple build is failing because it can't find libXNNPACK.a: https://github.com/pytorch/executorch/actions/runs/16251337270/job/45881437516?pr=12425#step:9:47386

android build is failing because I didn't propagate cross compilation settings to the ExternalProject: https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881440178?pr=12425#step:15:16927

llama runner linux build is failing because CMake is passing -lXNNPACK instead of the path to the static lib. (this happened during debugging before and it was because the XNNPACK target wasn't defined; probably the same again?) https://github.com/pytorch/executorch/actions/runs/16251337288/job/45881439862?pr=12425

unittest failures don't look related but we'll want to keep an eye on them.

[swolchok]

I want to see EXPORT work locally before I come back and fix broader platform support, so I'm going to do that first. Converting to draft in the meantime.

[swolchok]

build_frameworks_ios error:

2025-07-15T00:24:13.6245540Z Merging libraries:
2025-07-15T00:24:13.6245660Z libXNNPACK.a
2025-07-15T00:24:13.6245770Z libxnnpack_backend.a
2025-07-15T00:24:13.6245920Z libxnnpack-microkernels-prod.a
2025-07-15T00:24:13.6246060Z from ios/Release
2025-07-15T00:24:13.6246360Z into library /Users/runner/work/executorch/executorch/pytorch/executorch/cmake-out/libbackend_xnnpack_ios.a
2025-07-15T00:24:13.6246770Z File ios/Release/libXNNPACK.a does not exist

[swolchok]

test-llava-runner-linux failure looks like a flake: https://github.com/pytorch/executorch/actions/runs/16301129189/job/46035741014?pr=12425#step:15:15293 . therefore publishing for review

[mcr229]

changes in the XNNPACK cmake/dependencies.cmake look good to me

[mcr229]

curious what does this rearrangment do?

[swolchok]

I just found it weird that we were mutating xnnpack_third_party in cmake/Dependencies.cmake rather than setting it all at once

[mcr229]

oh shoot does this mean that any targets that link XNNPACK will also automatically link xnnpack-microkernels-prod as well? So we don't have to explicitly list both XNNPACK and xnnpack-microkernels-prod? I was trying to figure out how to do this for a while, but never figured it out.

[swolchok]

target_link_libraries with visibility PUBLIC or INTERFACE does this for normal libraries, but you need to do it this way for IMPORTED libraries.

[mcr229]

i think i merged something that might have conflict here. I just added a check

if (TARGET kleidiai)

before adding the library

[swolchok]

should be fine I think

[swolchok]

bunch of flakes that need to be rerun to make sure but no actual problems in the 7 failures so far

[swolchok]

Apparently FetchContent is preferred over ExternalProject. The thing I wanted from ExternalProject is separating our build from the dependency's build (i.e., not using add_subdirectory), because the XNNPACK dependency ecosystem doesn't support EXPORT. IIUC, we can get that separation using FetchContent by configuring it to support find_package (using either FIND_PACKAGE_ARGS when we don't require specific changes to or a bleeding-edge version of the dependency, or OVERRIDE_FIND_PACKAGE when we do have such a requirement; both will result in later find_package calls hitting the FetchContent version of the dep, it's just that FIND_PACKAGE_ARGS might find the thing installed system-wide) and then using find_package.