Skip to content

[mono][interp] Intrinsify Vector128.Store/StoreUnsafe on wasm - #132502

Merged
lewing merged 1 commit into
mainfrom
lewing-interp-vector128-store
Aug 20, 2026
Merged

[mono][interp] Intrinsify Vector128.Store/StoreUnsafe on wasm#132502
lewing merged 1 commit into
mainfrom
lewing-interp-vector128-store

Conversation

@lewing

@lewinglewing commented Aug 19, 2026

Copy link
Copy Markdown
Member

Rebased onto main now that #132500 has merged; this is a single commit touching 3 files.

Problem

Vector128.Store and Vector128.StoreUnsafe were deliberately left out of the PackedSimd alias table because their operands are reversed relative to the method they would lower to:

PackedSimd.Store (T* address, Vector128<T> source)
Vector128.Store (this Vector128<T> source, T* destination)

The alias path only renames the method, and emit_common_simd_epilogue assigns sregs in signature order, so aliasing them as-is would have passed the vector where the destination address is expected. They therefore fell back to managed code, which routes through Unsafe.WriteUnaligned — an intrinsic the interpreter does not implement — making a documented-as-fast API roughly six times slower than the PackedSimd equivalent.

Fix

Add them to the alias table and swap the two sregs after the epilogue has run. sregs are var indices consumed positionally by MINT_SIMD_INTRINS_P_PP (interp.c:6141), so the swap is a permutation: it changes their order, not the set of vars used, leaving liveness and refcounting unaffected. It runs during IL→IR generation, before every optimization pass.

It also matches what the jiterpreter already expects — SimdIntrinsic3.StoreANY in jiterpreter-trace-generator.ts loads arg 2 as the address and arg 3 as the vector.

Store is registered for every element type (ANY), so the reorder is guarded on the shape actually being a store: two parameters, void return, and a raw address (T* or ref T) as the second parameter. The three-argument StoreUnsafe(source, destination, elementOffset) has no PackedSimd counterpart and is rejected by the parameter count, falling back to managed code as before.

System.Numerics.Vector.Store/StoreUnsafe share the identical shape and route through the same emit path, so they are lowered too. StoreAligned/StoreAlignedNonTemporal are absent from the alias table and continue to fall back to managed code — importantly, since StoreAligned has a runtime alignment check that must throw.

Alignment

interp_packedsimd_store assigned through a v128_t*, claiming 16-byte alignment that neither PackedSimd.Store nor Vector128.StoreUnsafe guarantee. It now uses wasm_v128_store, which stores through a __packed__ __may_alias__ struct. This mirrors interp_packedsimd_load128's existing use of wasm_v128_load, and matches the jiterpreter's v128_store with an alignment hint of 1. Misaligned access is well-defined at the wasm ISA level, so this was C UB rather than a live bug.

Measurements

browser-wasm under V8, interpreter, 1,000,000 iterations, 2 warmups, best of 5, same harness before and after on the same machine:

OperationBeforeAfterSpeedup
Vector128.Store<int>3.243 ns0.521 ns6.2x
Vector128.StoreUnsafe<int>3.253 ns0.518 ns6.3x
PackedSimd.Store<int> (control)0.485 ns0.527 nsunchanged

The control's ~8% drift bounds run-to-run noise; the 6x is far outside it. No loop-overhead baseline is subtracted — an empty-loop control measured higher than the store loops, since the jiterpreter doesn't trace it.

Validation

  • browser-wasm mono+libs Release: 0 errors, 0 warnings
  • native osx-arm64 mono Release: 0 errors, 0 warnings
  • System.Runtime.Intrinsics: 13024/13024 passing (13021 baseline + 3 new)
  • System.Numerics.Vectors: 7449/7449 passing
  • IR (MONO_VERBOSE_METHOD): simd_intrins_p_pp [74 <- 73 72], 191 — descending sregs confirm the swap; the three-argument overload still emits a managed call
  • System.Numerics.Vector<T>.StoreUnsafe likewise emits simd_intrins_p_pp [19 <- 18 14], 191

Tests

Three tests covering Store, StoreUnsafe, and StoreUnsafe with an element offset, across twelve element types, at every 16-byte alignment, with guard bytes on both sides so a misplaced or oversized store is caught. The element-offset arm uses StoreUnsafe(ref *(destination - 1), elementOffset: 1) so a dropped offset argument shows up as a misplaced store. They pass on an unmodified runtime as well, so they validate behavior rather than implementation.

Note

This pull request was authored with the assistance of GitHub Copilot.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @vitek-karas, @BrzVlad, @kotlarmilos
See info in area-owners.md if you want to be subscribed.

@lewinglewing modified the milestones: 10.0.x, 11.0.0Aug 19, 2026
Base automatically changed from lewing-fix-packedsimd-alias-table to mainAugust 19, 2026 12:26
@lewing
lewingforce-pushed the lewing-interp-vector128-store branch from 0a29d41 to 2e98376CompareAugust 19, 2026 12:26
Vector128.Store and Vector128.StoreUnsafe were left out of the PackedSimd
alias table because their operands are reversed relative to the PackedSimd
method they would lower to:
PackedSimd.Store (T* address, Vector128<T> source)
Vector128.Store (this Vector128<T> source, T* destination)
The alias path only renames the method, and emit_common_simd_epilogue
assigns sregs in signature order, so aliasing them as-is would have passed
the vector where the destination address is expected. They therefore fell
back to managed code, which routes through Unsafe.WriteUnaligned - an
intrinsic the interpreter does not implement - making a documented-as-fast
API roughly six times slower than the PackedSimd equivalent.
Add them to the alias table and swap the two sregs after the epilogue has
run. sregs are var indices consumed positionally by MINT_SIMD_INTRINS_P_PP,
so the swap is a permutation: it changes their order, not the set of vars
used, leaving liveness and refcounting unaffected. It also matches what the
jiterpreter already expects - SimdIntrinsic3.StoreANY loads arg 2 as the
address and arg 3 as the vector.
Store is registered for every element type, so guard the reorder on the
shape actually being a store: two parameters, void return, and a raw
address (T* or ref T) as the second parameter. The three-argument
StoreUnsafe (source, destination, elementOffset) has no PackedSimd
counterpart and is rejected by the parameter count, falling back to
managed code as before.
Also fix interp_packedsimd_store to use wasm_v128_store instead of
assigning through a v128_t*, which claimed 16-byte alignment that neither
PackedSimd.Store nor Vector128.StoreUnsafe guarantee. This mirrors
interp_packedsimd_load128, which already uses wasm_v128_load, and matches
the jiterpreter's v128_store with an alignment hint of 1. Misaligned
access is well-defined at the wasm ISA level, so this was C UB rather than
a live bug.
System.Numerics.Vector.Store/StoreUnsafe share the identical shape and
route through the same emit path, so they are lowered too;
StoreAligned/StoreAlignedNonTemporal are absent from the alias table and
continue to fall back to managed code.
Add tests covering Store, StoreUnsafe, and StoreUnsafe with an element
offset across twelve element types at every 16-byte alignment, with guard
bytes on both sides so a misplaced or oversized store is caught.
Measured on browser-wasm under V8, 1,000,000 iterations, best of 5:
Vector128.Store<int> 3.243 ns -> 0.521 ns (6.2x)
Vector128.StoreUnsafe<int> 3.253 ns -> 0.518 ns (6.3x)
PackedSimd.Store<int> 0.485 ns -> 0.527 ns (control, unchanged)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3ebb2f49-c9fc-4c28-9755-c51b9deac735
CopilotAI lite review requested due to automatic review settings August 19, 2026 14:25
@lewing
lewingforce-pushed the lewing-interp-vector128-store branch from 2e98376 to 2e58d7aCompareAugust 19, 2026 14:25

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables Mono interpreter SIMD lowering for Vector128.Store / Vector128.StoreUnsafe (and the analogous System.Numerics.Vector APIs) on wasm by routing them through the existing PackedSimd lowering path while correcting for the reversed operand order, and fixes the wasm store helper to avoid C UB on unaligned destinations.

Changes:

  • Add Store / StoreUnsafe to the PackedSimd alias table and swap the two operand sregs after emit_common_simd_epilogue assigns them in signature order.
  • Update interp_packedsimd_store to use wasm_v128_store (unaligned-safe) instead of assigning through a v128_t*.
  • Add new Vector128 tests covering Store, StoreUnsafe, and StoreUnsafe with element offset across multiple element types and destination alignments.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

FileDescription
src/mono/mono/mini/interp/transform-simd.cEnables aliasing of Store/StoreUnsafe and performs a post-epilogue sreg swap so the PackedSimd helper receives (address, vector) ordering.
src/mono/mono/mini/interp/interp-simd.cReplaces a potentially-alignment-assuming store through v128_t* with wasm_v128_store for well-defined unaligned stores.
src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csAdds regression tests validating correct store placement and size across alignments, types, and the element-offset overload behavior.

@lewing
lewing merged commit be1c680 into mainAug 20, 2026
104 of 106 checks passed
@lewing
lewing deleted the lewing-interp-vector128-store branch August 20, 2026 13:34
@lewing

Copy link
Copy Markdown
MemberAuthor

/backport to release/11.0-rc1

@github-actions

Copy link
Copy Markdown
Contributor

Started backporting to release/11.0-rc1 (link to workflow run)

@github-actions

Copy link
Copy Markdown
Contributor

@lewing backporting to release/11.0-rc1 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output
$ git cherry-pick be1c680101458a735200e68a74460919be249315Auto-merging src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csCONFLICT (content): Merge conflict in src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csAuto-merging src/mono/mono/mini/interp/interp-simd.cAuto-merging src/mono/mono/mini/interp/transform-simd.cCONFLICT (content): Merge conflict in src/mono/mono/mini/interp/transform-simd.cerror: could not apply be1c6801014... [mono][interp] Intrinsify Vector128.Store/StoreUnsafe on wasm (#132502)hint: After resolving the conflicts, mark them withhint: "git add/rm <pathspec>", then runhint: "git cherry-pick --continue".hint: You can instead skip this commit with "git cherry-pick --skip".hint: To abort and get back to the state before "git cherry-pick",hint: run "git cherry-pick --abort".hint: Disable this message with "git config set advice.mergeConflict false"
$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patchApplying: [mono][interp] Intrinsify Vector128.Store/StoreUnsafe on wasmUsing index info to reconstruct a base tree...M	src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csM	src/mono/mono/mini/interp/interp-simd.cM	src/mono/mono/mini/interp/transform-simd.cFalling back to patching base and 3-way merge...Auto-merging src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csCONFLICT (content): Merge conflict in src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.csAuto-merging src/mono/mono/mini/interp/interp-simd.cAuto-merging src/mono/mono/mini/interp/transform-simd.cCONFLICT (content): Merge conflict in src/mono/mono/mini/interp/transform-simd.cerror: Failed to merge in the changes.hint: Use 'git am --show-current-patch=diff' to see the failed patchhint: When you have resolved this problem, run "git am --continue".hint: If you prefer to skip this patch, run "git am --skip" instead.hint: To restore the original branch and stop patching, run "git am --abort".hint: Disable this message with "git config set advice.mergeConflict false"Patch failed at 0001 [mono][interp] Intrinsify Vector128.Store/StoreUnsafe on wasmError: The process '/usr/bin/git' failed with exit code 128

Link to workflow output

lewing added a commit that referenced this pull request Aug 20, 2026
…afe on wasm (#132575)
Manual backport of #132502 to `release/11.0-rc1`.
## Why this needed a manual backport
The automatic cherry-pick conflicted in two files:
- **`transform-simd.c`** — resolved by first landing the prerequisite,
#132525 (backport of #132500). Once that merged, this file auto-merged
with no hand edits.
- **`Vector128Tests.cs`** — a pure placement conflict. #132502's tests
were appended after `Vector128GetElementVariableOutOfRangeTest`, which
comes from #132499 and is not on this branch. The three Store tests are
self-contained, so they were appended at the end of the class instead.
No #132499 tests were pulled in.
Every added and removed line in this commit is byte-identical to
upstream `be1c680` — only hunk headers and line numbers differ.
## Depends on #132525
This backport is only effective because #132525 already merged. On
`release/11.0-rc1` before that fix, `packedsimd_alias_methods` was
mis-sorted, and since `lookup_intrins()` binary-searches the table,
`Store`, `StoreUnsafe`, `Subtract` and `SubtractSaturate` were all
unreachable. Simulating `mono_binary_search` against the pre-#132525
table confirms this — so #132502 on its own would have been a silent
no-op.
With #132525 merged, the resulting table is strictly sorted and all 49
entries resolve; `Store` and `StoreUnsafe` land at indices 29 and 30.
## Original change
`PackedSimd.Store(T* address, Vector128<T> source)` takes its operands
in the opposite order from `Vector128.Store(this Vector128<T> source, T*
destination)`, and the alias path only renames the method —
`emit_common_simd_epilogue` assigns sregs in signature order. This adds
them to the alias table and swaps the two sregs after the epilogue,
guarded on the shape actually being a store (two parameters, void
return, raw address as the second parameter). The three-argument
`StoreUnsafe(source, destination, elementOffset)` has no PackedSimd
counterpart and still falls back to managed code.
`interp_packedsimd_store` also switches from assigning through a
`v128_t*` to `wasm_v128_store`, which does not claim 16-byte alignment
that neither API guarantees.
Upstream measured ~6.2x on `Vector128.Store<int>` and ~6.3x on
`Vector128.StoreUnsafe<int>` under the interpreter.
## Local validation
- `./build.sh clr+libs -rc release` — 0 errors, 0 warnings
- `System.Runtime.Intrinsics.Tests` — **13011/13011 passing, 0 failed**,
including the three new tests: `Vector128StoreUnalignedTest`,
`Vector128StoreUnsafeUnalignedTest`,
`Vector128StoreUnsafeElementOffsetUnalignedTest`
The count is 13011 rather than upstream's 13024 because the #132499 and
#132496 tests are not on this branch.
Not validated locally: the `interp-simd.c` change is wasm-only, so a
native osx-arm64 build does not exercise it. It is byte-identical to the
upstream commit, which passed full CI.
> [!NOTE]
> This pull request was authored with the assistance of GitHub Copilot.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3ebb2f49-c9fc-4c28-9755-c51b9deac735
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@lewing@BrzVlad