arm64: Optimize ARM64 compare mask ExtractMostSignificantBits consumers

[jonathandavies-arm]

See discussion at #121981 (comment)

Diffs

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[EgorBo]

Isn't it the same as #126790 ?
The difference is that doing everything in Rationalizer like this PR does will not allow to see that the input is 0/allbits sets unless it's directly propagated as a tree.

[a74nh]

Isn't it the same as #126790 ? The difference is that doing everything in Rationalizer like this PR does will not allow to see that the input is 0/allbits sets unless it's directly propagated as a tree.

What's the status on #126790 ? It hasn't moved since April. Is it planned to get in for NET11 ?

[EgorBo]

Isn't it the same as #126790 ? The difference is that doing everything in Rationalizer like this PR does will not allow to see that the input is 0/allbits sets unless it's directly propagated as a tree.

What's the status on #126790 ? It hasn't moved since April. Is it planned to get in for NET11 ?

@tannergooding Optimized all Vector*.IndexOf* with SHRN on the C# side in this PR: #126678 and my PR basically became useless, see benchmark: #126790 (comment)

We still have a few places where we call Compare + EMSB instead of explicit IndexOf APIs, that is what my other PR tries to do: #126841

There will be a few places where we won't be able to replaces EMSB with IndexOf (e.g. iterators over multiple matches), though. But I don't think this PR does a good job for them either judging by the diffs -

Basically, +600 LOC of JIT changes for 1 use-case (context) in benchmarks.

[EgorBo]

If you want to make this transformation more useful, you need to borrow the assertionprop.cpp changes from #126790 - AllComponentsEitherZeroOrAllBitsSet so it can leave a GTF_* flag on a tree you handle in the rationalizer. Because expecting the input to be a comparison tree is a very conservative impl (hence, small diffs).

[EgorBo]

A lot bigger diffs now.

[EgorBo]

LGTM, thanks. Perhaps @tannergooding @dotnet/jit-contrib want to take a look at the rationilizer impl?

[EgorBo]

@jonathandavies-arm CI build failures are related

[github-actions]

Workflow state for the Holistic Review Orchestrator.

{
"version": 5,
"last_dispatched_commit": "acabdd938d618f866d47deb0b29e52a70d7fdf5e",
"last_dispatched_base_ref": "main",
"last_dispatched_base_sha": "1d85f408cee362c3b693e8b2ccba114b976b317d",
"last_reviewed_commit": "acabdd938d618f866d47deb0b29e52a70d7fdf5e",
"last_reviewed_base_ref": "main",
"last_reviewed_base_sha": "1d85f408cee362c3b693e8b2ccba114b976b317d",
"last_recorded_worker_run_id": "29681922766",
"review_attempt_commit": "",
"review_attempt_base_ref": "",
"review_attempt_count": 0,
"max_review_attempts": 5,
"review_history_format": "holistic-review-disclosure-v1",
"review_history": [
{
"commit": "acabdd938d618f866d47deb0b29e52a70d7fdf5e",
"review_id": 4730551207
}
]
}

[github-actions]

Holistic Review

Motivation: Justified. On ARM64 Vector*.ExtractMostSignificantBits has no single instruction and is emulated with a long sequence; when the input is a comparison mask (all-bits-set or zero per element) and the extracted bitmask is only consumed by == 0/!= 0, PopCount, or Leading/TrailingZeroCount, the full bitmask never needs to be materialized. The SuperPMI diffs (-480 bytes, -28% PerfScore over 39 contexts) confirm a real win.

Approach: Sound and idiomatic for the JIT. It recognizes comparison masks via IsVectorPerElementMask (extended to accept vector compares where intrinsicSimdBaseType element size >= the requested base type, valid since a larger-element all-set/zero mask reinterpreted at a smaller element size is still a per-element mask), propagates the fact onto single-def locals through reaching-VN analysis in optAssertionProp_HWIntrinsic, and rewrites the consumer in rationalize into horizontal MaxAcross/AddAcross/MinAcross reductions (with pairwise fallbacks for the 2-lane Vector64<uint> case that lacks a 2S across form). I hand-checked the index/sentinel encodings for the zero-count rewrites (Trailing: index+1/sentinel 33, then - 1; Leading: 31-index/sentinel 32) and they reproduce the scalar TrailingZeroCount/LeadingZeroCount results, including the 32 zero-mask case.

Summary: ⚠️ Needs Human Review. I found no blocking correctness issues — the transforms, VN/local-mask propagation, and disasm/runtime test coverage all look correct. Because this is intricate ARM64 JIT lowering with subtle mask-width and reduction reasoning, a JIT maintainer should confirm the IsVectorPerElementMask widening rule and the local-mask metadata lifetime. See non-blocking notes below.

---

Detailed Findings

✅ Correctness of rewrites and mask recognition

The == 0/!= 0, PopCount, and Leading/TrailingZeroCount rewrites each verify IsHWIntrinsicCmpMaskExtractMsb before transforming, normalize the base type to unsigned, and cast the reduced scalar as unsigned to TYP_INT. The zero-count sentinel/index vectors correctly preserve the scalar semantics for the all-zero mask. gentree.cpp and valuenum.cpp keep their two IsVectorPerElementMask implementations in sync as the in-code comment requires.

✅ Relaxed assert in IsVectorPerElementMask

Replacing the assert(varTypeIsSIMD(...))/size asserts with early return false (gentree.cpp:33782) is appropriate now that callers pass through GT_LCL_VAR locals and reaching-VN types that may not match, avoiding false asserts while preserving the negative answer.

✅ Test quality

ExtractMostSignificantBits.cs combines runtime value assertions with ARM64-FULL-LINE disasm checks across byte/ushort/int and both Vector128 and Vector64, plus a ...ViaLocal variant that exercises the new single-def local-mask propagation path and an explicit zero-mask index case. This is strong, targeted coverage for the new paths.

💡 Duplicated pairwise-reduction scaffolding (follow-up, non-blocking)

The Vector64<uint> spill-clone-and-pairwise block (MakeDummyUse → ReplaceWithLclVar → gtClone → *Pairwise) is repeated almost verbatim in RewriteHWIntrinsicCmpMaskExtractMsb, ...PopCount, and ...ZeroCount, differing only by the reduction intrinsic. Extracting a small helper parameterized by the across/pairwise intrinsic pair would reduce the risk of the three copies drifting. Maintainability suggestion, not a blocker.

Note

This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.

Generated by Holistic Review · 192.2 AIC · ⌖ 15 AIC · ⊞ 10K

[EgorBo]

@tannergooding does it look good?

[tannergooding]

The changes overall look reasonable, but I had some questions and feedback.

[EgorBo]

• [Perf] Linux/arm64: 49 Improvements on 8/5/2026 1:23:13 AM +00:00 perf-autofiling-issues#77882