[MLAS] Add Apple Accelerate (vForce) Tanh kernel (depends on #32001)

[Justin Chu (justinchuby)]

Outcome / Decision: CLOSING — not merging

Decision: this PR will be closed, not merged. The real Apple Silicon Release benchmark below disproves the optimization premise this PR was built on: apple_accelerate_vforce (BM_TanhDispatch) is slower than the existing portable polynomial kernel (BM_TanhPortableKernel) at every one of the 18 sizes tested (N = 1, 15, 16, 31, 32, 63, 64, 127, 128, 255, 256, 511, 512, 1024, 4096, 16384, 65536, 262144) — honest range ~1.1x–5x slower, median ~2.2x slower. There is no size regime in the data where vForce wins. See the full per-size table in Validation below.

• Correctness is not in question. All 10 TanhAppleAccelerate.* tests passed on real Apple Silicon CI hardware (10/10, both Debug and Release matrix legs) — the kernel is correct, just not faster.
• Real hardware evidence, not CI noise. Run 31647787148: Release job (36m29s, correctness + the benchmark A/B) and Debug job (54m44s, full build.py --test suite). Both green. This was the first (and, per this decision, only) time any revision of this PR executed on real Apple hardware.
• Why close rather than leave open/draft: Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001 is an infrastructure-only umbrella option (onnxruntime_USE_APPLE_ACCELERATE) intended to let real Accelerate-backed kernels opt in behind it. Merging this PR as the option's first concrete kernel would make enabling that option a pessimization for Tanh — every caller on the ONNX Tanh op, the Gelu tanh-approximation, the SVM classifier, and RNN/GRU/LSTM cell helpers would get consistently slower code than the portable kernel already ships. That is not an acceptable trade for an opt-in performance option, and there is no evidence in hand (across 18 sizes, on real hardware) that a narrower size-gated variant would help either — the ratio is roughly flat across three orders of magnitude of N, not converging toward parity or a crossover at any tested size.
• What should be salvaged separately (not in this PR): the reusable test/CI methodology developed here is sound and worth keeping for the next Accelerate-backed kernel candidate (e.g. a future SGEMM/BNNS/vDSP PR building on Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001): the finite out-of-range poison-value technique for proving a kernel actually wrote every output element (kPoisonValue = 123456.75f, proven via an injected no-write bug under qemu-aarch64-static); the list-then-verify fail-loud CI pattern for both gtest (--gtest_list_tests) and benchmark (--benchmark_list_tests=true) steps, which catches a filter silently matching zero tests/benchmarks; the A/B benchmark pattern with an explicit dispatch-label assertion (apple_accelerate_vforce presence check) so a misconfigured build can't silently compare the portable kernel against itself; and the direct IEEE-754/ONNX-contract semantic tests for NaN/±Inf/±0/subnormal/FLT_MIN inputs, independent of parity against the portable kernel. None of this is Tanh-specific and all of it transfers to a kernel that might actually show a win.
• This PR itself, and its dependency Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001, remain unmerged; closing this PR does not require any changes to Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001.

Summary

First concrete kernel built on top of #32001 (onnxruntime_USE_APPLE_ACCELERATE, infrastructure-only, still open/unmerged). Adds MlasTanhKernelAppleAccelerate, which computes MLAS's Tanh activation via Apple Accelerate's vForce vvtanhf() on macOS arm64 only, gated entirely behind the (default-off) onnxruntime_USE_APPLE_ACCELERATE compile option added in #32001.

• Depends on Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001. This branch is stacked on nxrt/mlas-apple-framework-option and will need to be rebased once that PR merges. Do not review/merge this before Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001.
• Scope: macOS arm64 only. No Intel Mac, no universal2, no iOS/tvOS/visionOS. Matches the squad's standing Apple-scope directive.
• Public API only. vvtanhf(float* y, const float* x, const int* n) is a public, long-stable vForce API (macOS 10.4+ / iOS 5.0+) — verified against the actual Accelerate.framework headers, not just documentation. No private/hand-declared BNNS structs are used.
• One kernel family. Accelerate cblas SGEMM, BNNS, and vDSP are explicitly out of scope for this PR and will be separate follow-ups.

Why Tanh / vForce

Surveyed MLAS's elementwise/activation kernels (Erf, Tanh, Exp, Gelu) for the smallest, highest-value, provably-reachable candidate with a stable public API:

• vForce has no erf function (checked actual SDK headers through macOS 11.3), ruling out Erf despite it being MLAS's most self-contained kernel.
• vvtanhf is public and stable, and MlasComputeTanh is reachable from several real, high-value call sites: the ONNX Tanh activation op, the Gelu tanh-approximation, the SVM classifier, and RNN/GRU/LSTM cell helpers — several of which call it with Input == Output (in-place).
• On plain ARM64 (Apple Silicon has no SVE), MlasComputeTanh<float> currently bypasses the platform dispatch-table mechanism entirely and calls the generic kernel directly (the #if MLAS_TARGET_AMD64 || MLAS_TARGET_RISCV64 || MLAS_USE_SVE condition is false on plain ARM64) — so the Apple path is added as a new highest-priority compile-time branch in MlasComputeTanh<float>, not via the platform function-pointer table.

What's in this PR

1. Kernel (onnxruntime/core/mlas/lib/tanh.cpp, mlasi.h): MlasTanhKernelAppleAccelerate, chunking N defensively against INT32_MAX (since vvtanhf takes const int*), documented as safe for in-place use and free of internal GCD/thread-pool dispatch (so it does not oversubscribe the ORT threadpool). Wired into MlasComputeTanh<float> as the first-priority branch, behavior-neutral (default OFF) everywhere else.

2. Tests (onnxruntime/test/mlas/unittest/test_tanh_apple_accelerate.cpp, gated identically to the kernel), 10 tests total:

   • ForcedReachability -- calls MlasTanhKernelAppleAccelerate directly (not through dispatch), proving the symbol links/is callable whenever the option is on, independent of dispatch wiring.
   • InPlaceAliasing -- Input == Output, matching real ORT call sites (Gelu tanh-approx, RNN/GRU/LSTM cells).
   • PublicDispatchMatchesDirectKernelCall -- asserts MlasComputeTanh<float> actually resolves to the direct kernel call (catches dispatch/kernel drift).
   • LargeBufferManyVectorIterations -- 1Mi elements, exercises the INT32_MAX-defensive chunking loop (vvtanhf takes const int*).
   • SpecialValuesSemanticsSingleElement / ...ScalarTailBuffer -- assert the IEEE-754/ONNX Tanh contract directly (not portable-kernel parity) for NaN (NaN in -> NaN out, payload/sign unasserted), +/-Inf (-> exactly +/-1.0f), and signed +/-0.0 (-> +/-0.0f, sign bit preserved exactly).
   • DenormalAndNearZeroSemanticsSingleElement / ...ScalarTailBuffer -- true subnormal inputs only (denorm_min()), asserted via an either/or check tolerant of both IEEE-correctly-rounded (tanh(x)~=x) and flush-to-zero (FTZ) hardware behavior: never NaN, sign preserved for a nonzero result, |output| <= |input|. Deliberately does not assert which behavior real vForce exhibits (unverifiable without hardware); the actual value is printed for a human to observe.
   • SmallestNormalSemanticsSingleElement / ...ScalarTailBuffer -- new this revision: +/-FLT_MIN (smallest positive normal float, not subnormal) split out from the denormal tests above and asserted strictly/exactly (output == input, sign preserved) rather than the FTZ-tolerant either/or check, since FTZ is not a legitimate rationale for a normal input. Verified numerically that tanh(FLT_MIN) rounds to exactly FLT_MIN in float32 (the -x^3/3 correction term is many orders of magnitude below one ULP).

   Every test pre-fills its output buffer with a finite, out-of-range poison sentinel (kPoisonValue = 123456.75f), not 0.0f and not NaN, before calling the kernel under test. tanh's range is always [-1, 1], so this value can never be a genuinely correct output at any tested input, and a kernel that silently fails to write an element (e.g. an off-by-one in the scalar-tail path) cannot pass by coincidence. A NaN poison was tried first and had a real hole: for NaN inputs, the correct contract is NaN output, so an unwritten NaN-poisoned buffer would satisfy std::isnan() and pass even though the kernel never wrote anything -- silently defeating the poison for exactly the inputs it most needs to guard. This was proven concretely (not just reasoned about) by cross-compiling the real kernel+test for aarch64-linux-gnu, injecting a deliberate no-write bug into a stub vvtanhf, and running under qemu-aarch64-static: the old NaN poison reported 10/10 "PASSED" (silent), the new finite poison correctly failed 2 tests / 4 assertions.

3. Benchmarks (onnxruntime/test/mlas/bench/bench_transcendental.cpp), following the existing BM_SiluDispatch/BM_GeluErfDispatchExact A/B pattern:

   • BM_TanhDispatch -- calls the public MlasComputeTanh<float> entry point (whatever it actually resolves to on this build), labeled via GetTanhDispatchPathInfo() as "apple_accelerate_vforce" when the option is enabled.
   • BM_TanhPortableKernel -- calls MlasTanhKernel (the portable polynomial reference) directly as a fixed baseline, labeled "portable_polynomial" unconditionally, regardless of platform.
   • The two distinct labels in the printed output are what prove a given run exercised the intended A/B and not two copies of the same kernel under different names -- this is mechanically checked by CI, not just asserted in prose (see CI below).

4. CI (mac.yml, macos-ci-build-and-test-workflow.yml): new apple_accelerate job, forwarding --use_apple_accelerate (the build.py flag Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001 already added).

   • Matrix: both arm64/Debug (assertion-rich correctness coverage, NDEBUG off) and arm64/Release. Debug runs the full build.py --test suite plus the targeted Tanh tests; the benchmark build/run steps below are gated to build_config == 'Release' only, since a Debug/-O0 binary is not meaningful performance evidence.
   • Targeted test step prints Build config: <config> then runs onnxruntime_mlas_test --gtest_filter="TanhAppleAccelerate.*" --gtest_list_tests first and fails loudly (::error::, exit 1) if the listing is empty -- a gtest binary invoked with a filter matching zero tests still exits 0, so this step would otherwise appear to pass while providing zero real coverage (e.g. if MLAS_USE_APPLE_ACCELERATE/__APPLE__/MLAS_TARGET_ARM64 didn't all evaluate true on this job). Only after that check passes does it actually run the filtered tests.
   • Benchmark build step (Release only) does not run build.py --build ${{ env.build_flags }} as one command: build_flags includes --build_wheel, and build.py's main() unconditionally repackages the Python wheel whenever args.build and args.build_wheel are both set, regardless of --target -- so a naive "add --build_micro_benchmarks --target onnxruntime_mlas_benchmark" would rebuild/repackage the entire wheel just to add one benchmark binary. Instead: build.py --update --build_micro_benchmarks ${{ env.build_flags }} (reconfigure only, no --build, so it adds --build_micro_benchmarks into the existing CMake cache in-place without touching the wheel path or discarding the build tree), then cmake --build ./build/<config> --config <config> --target onnxruntime_mlas_benchmark --parallel directly.
   • Benchmark run step (Release only) prints Build config: <config>, then (matching the test step's zero-match pattern) runs --benchmark_filter="BM_Tanh(Dispatch|PortableKernel)" --benchmark_list_tests=true first and fails loudly if either BM_TanhDispatch or BM_TanhPortableKernel is absent from the listing (a benchmark filter matching zero benchmarks also exits 0 with an empty table). Only then does it run the actual A/B, print the output, and fail loudly if the apple_accelerate_vforce label is absent from that output -- guarding against a misconfigured build where BM_TanhDispatch silently fell back to the portable kernel, which would otherwise print two portable_polynomial-labeled results as if they were a real A/B.
   • Result: both matrix entries are green on real Apple Silicon CI runners as of this revision (see Validation below for run links and actual output).

Validation performed

This revision was pushed to real macOS arm64 Apple Silicon CI runners on GitHub Actions (apple_accelerate job, both Debug and Release matrix entries), in addition to the static/cross-compiled validation from earlier revisions (summarized below). Both jobs are green:

• Release: https://github.com/microsoft/onnxruntime/actions/runs/31647787148/job/94285366590 (36m29s)
• Debug: https://github.com/microsoft/onnxruntime/actions/runs/31647787148/job/94285366640 (54m44s)

This is the first time any revision of this PR has actually executed on real Apple hardware. Two real build-break bugs were found and fixed in the process (both were pre-existing in tanh.cpp, not previously exercised because no prior revision had ever run on a real macOS arm64 runner with --use_apple_accelerate):

• The Xcode 26.3 / MacOSX26.2 SDK's <Accelerate/Accelerate.h> umbrella header fails to compile under C++ (ISO C++ forbids forward references to 'enum' types in vecLib's cblas.h/cblas_new.h -- both BLAS header variants are affected on this SDK, a known-class Apple SDK header bug unrelated to this kernel).
• Fixed by forward-declaring the one vForce symbol this kernel actually needs (extern "C" void vvtanhf(float*, const float*, const int*), matching its long-stable public signature) instead of including the umbrella header at all. The Accelerate framework is already linked via cmake/onnxruntime_mlas.cmake, so the symbol resolves at link time unchanged.

Real correctness results (Release job, onnxruntime_mlas_test --gtest_filter='TanhAppleAccelerate.*'): all 10 tests passed, including the two new SmallestNormalSemantics* tests added this revision:

[==========] Running 10 tests from 1 test suite.
...
[  PASSED  ] 10 tests.

The Debug job's full build.py --test run (entire test suite, not just the Tanh-specific filter) also passed.

Real Release benchmark A/B results (onnxruntime_mlas_benchmark, on the CI runner's 3-core host) -- reported here in full, honestly, including the outcome. All 18 registered sizes, with the computed ratio (Dispatch/Portable, i.e. vForce relative to the portable kernel — >1.0 means vForce is slower):

N	BM_TanhDispatch (vForce)	BM_TanhPortableKernel (portable)	ratio
1	20.5 ns	4.08 ns	5.02x slower
15	19.3 ns	17.3 ns	1.12x slower
16	18.2 ns	9.22 ns	1.97x slower
31	35.2 ns	29.4 ns	1.20x slower
32	36.6 ns	17.1 ns	2.14x slower
63	69.7 ns	40.3 ns	1.73x slower
64	78.4 ns	35.3 ns	2.22x slower
127	144 ns	87.9 ns	1.64x slower
128	177 ns	57.2 ns	3.10x slower
255	292 ns	124 ns	2.35x slower
256	299 ns	136 ns	2.20x slower
511	574 ns	262 ns	2.19x slower
512	557 ns	269 ns	2.07x slower
1024	1308 ns	597 ns	2.19x slower
4096	4537 ns	2059 ns	2.20x slower
16384	18496 ns	8148 ns	2.27x slower
65536	67639 ns	33355 ns	2.03x slower
262144	309936 ns	135697 ns	2.28x slower

(Full raw output, including bytes_per_second/items_per_second columns, in the Release job log linked above.) The CI fail-loud checks described under "What's in this PR" above (list-then-verify for both benchmark names, apple_accelerate_vforce label presence) all passed as designed on this real run -- this is a genuine, verified A/B, not two portable-kernel runs masquerading as one.

Honest conclusion: apple_accelerate_vforce (BM_TanhDispatch) is consistently and substantially slower than the existing portable polynomial kernel (BM_TanhPortableKernel) at all 18 sizes tested, with no crossover and no size regime where vForce wins. Ratio ranges from ~1.1x slower (N=15) to ~5x slower (N=1), median ~2.2x slower; the bulk of the mid-to-large sizes (N ≥ 256) cluster tightly around 2.0x–2.3x slower, so this is not primarily a small-N dispatch-overhead artifact. No performance win is being claimed by this PR, and the data available does not support one. This is a single run on shared/virtualized CI hardware (3 vCPUs, Unable to determine clock rate from sysctl), so absolute ceilings on dedicated Apple Silicon hardware could differ in magnitude — but the direction and rough size of the effect (a consistent ~2x-order-of-magnitude penalty across three orders of magnitude of N) is unlikely to invert into a win on different real hardware. Combined with the absence of any favorable data point anywhere in the tested range, this is the basis for the decision above to close rather than merge or continue iterating on this kernel.

Additional validation performed before/alongside the real CI run above:

• Both modified GitHub Actions workflow YAML files validated with yaml.safe_load (no syntax errors).
• clang-format --style=file --dry-run --Werror clean on the modified test file (pre-existing, unrelated formatting deviations in tanh.cpp were confirmed present before this revision's changes too, via git stash diff).
• The exact bash logic in the CI steps described above (zero-match list-then-verify for both the gtest and benchmark steps, missing-benchmark-name failure, missing-apple_accelerate_vforce-label failure) was extracted and run locally against stub onnxruntime_mlas_test/onnxruntime_mlas_benchmark binaries (success, zero-match, missing-name, missing-label cases) before ever reaching real CI -- all behaved as intended, giving confidence in the design ahead of the real run that then confirmed it end-to-end above.
• Cross-architecture proof for the no-write poison fix (design explained under "Tests" above): cross-compiled the real, unmodified test file for aarch64-linux-gnu with __APPLE__/MLAS_USE_APPLE_ACCELERATE forced, ran under qemu-aarch64-static with a deliberately injected no-write bug -- silently undetected (10/10 "PASSED") against the old NaN poison, correctly caught (2 failed, 4 assertions) against the new finite poison. This is what the real hardware run above then re-confirmed did not regress (all 10 tests still pass for real).

Status of the pre-close blockers (for the record)

• Real Apple Silicon numeric validation — done. All 10 tests (including the new SmallestNormalSemantics* tests) pass against real vvtanhf on real Apple Silicon CI hardware (see Validation above). The finite no-write poison and ±FLT_MIN/denormal separation both executed against real hardware for the first time and found no correctness issues.
• Real Apple Silicon Release benchmark numbers — done, and the result is unfavorable at every tested size: apple_accelerate_vforce was slower than portable_polynomial across all 18 sizes tested on the CI runner, ~1.1x–5x slower, median ~2.2x (see Validation above for the full per-size table and run link).
• CI green on the new apple_accelerate macOS arm64 job — both Debug and Release matrix entries pass (run 31647787148).
• Re-evaluate this PR's premise given the real benchmark result — resolved: closing. The motivating assumption (vForce's vvtanhf is faster than MLAS's portable polynomial Tanh) is contradicted by real hardware data at every size tested, with no crossover point and no favorable regime. See the Outcome / Decision section at the top of this description for the full reasoning and what should be salvaged separately.
• Rebase-onto-Add onnxruntime_USE_APPLE_ACCELERATE CMake option (infrastructure only) #32001 and ORT-threadpool-interaction items are moot given the close decision and are not being pursued further in this PR.