Uh oh!
There was an error while loading. Please reload this page.
JIT: less loop inversion for bottom tested loops with no evident IV - #130368
JIT: less loop inversion for bottom tested loops with no evident IV#130368AndyAyersMS wants to merge 10 commits into
Conversation
Under JitLoopInversionRequireBenefitForBottomTested (off by default), skip inverting an already bottom-tested loop with no recognized IV unless the duplicated condition holds a call or a loop-invariant, hoistable load. Targets the arm64 regressions in dotnet#130045 while keeping the Dictionary Enumerator.MoveNext inversion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
AndyAyersMS
commented
Jul 8, 2026
@EgorBot -arm64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 usingSystem;usingSystem.Numerics;usingBenchmarkDotNet.Attributes;publicclassBench{privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintSpan_IndexOf_NotFound()=>_a.AsSpan().IndexOf(-1);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);[Benchmark]publicboolBigInteger_Equals()=>_x.Equals(_y);}Base is Note Comment generated with GitHub Copilot CLI. |
There was a problem hiding this comment.
Pull request overview
This PR adds an (opt-in) heuristic gate to optTryInvertWhileLoop so that when a loop is already bottom-tested and AnalyzeIteration recognizes no IV, loop inversion is only performed if duplicating the condition block appears to have a “benefit” (call or potentially hoistable memory load). It also introduces a new JIT config knob to enable this gate.
Changes:
- Track “exiting cond latch seen” separately from “IV-test latch” and only apply the new gate in the bottom-tested + no-IV case.
- Classify
condBlockfor calls/indirections and use the existing loop size walk to conservatively detect stores to locals used in indirection address expressions. - Add
JitLoopInversionRequireBenefitForBottomTestedconfig (default off).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
src/coreclr/jit/optimizer.cpp | Adds the benefit-based gate and condition classification for bottom-tested no-IV loops. |
src/coreclr/jit/jitconfigvalues.h | Introduces the new release config switch controlling the gate. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
AndyAyersMS
commented
Jul 8, 2026
Re-running on server arm64 (Cobalt 100) — the earlier @EgorBot -linux_arm64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 usingSystem;usingSystem.Numerics;usingBenchmarkDotNet.Attributes;publicclassBench{privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintSpan_IndexOf_NotFound()=>_a.AsSpan().IndexOf(-1);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);[Benchmark]publicboolBigInteger_Equals()=>_x.Equals(_y);}Base is Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 8, 2026
Adding the two biggest #130045 regressions ( @EgorBot -linux_arm64 -ubuntu24_azure_genoa -windows_x64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 usingSystem;usingSystem.Buffers;usingSystem.Collections.Generic;usingSystem.Linq;usingSystem.Numerics;usingSystem.Runtime.CompilerServices;usingBenchmarkDotNet.Attributes;publicclassBench{privateSearchValues<char>_searchValues;privatechar[]_textExcept;privateint[]_found;privateDictionary<int,int>_dictionary;privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){_searchValues=SearchValues.Create("ßäöüÄÖÜ");_textExcept=newstring('ß',256).ToCharArray();_textExcept[128]='\n';_found=Enumerable.Range(0,512).ToArray();_dictionary=_found.ToDictionary(k =>k, k =>k);for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicintLastIndexOfAnyExcept()=>_textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);[Benchmark]publicboolContainsKeyTrue_IDictionary()=>ContainsKey(_dictionary);[MethodImpl(MethodImplOptions.NoInlining)]privateboolContainsKey(IDictionary<int,int>collection){boolresult=false;varfound=_found;for(inti=0;i<found.Length;i++)result^=collection.ContainsKey(found[i]);returnresult;}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);}Base is Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 8, 2026
Also on Ampere (Neoverse-N1) — the core class behind the @EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 usingSystem;usingSystem.Buffers;usingSystem.Collections.Generic;usingSystem.Linq;usingSystem.Numerics;usingSystem.Runtime.CompilerServices;usingBenchmarkDotNet.Attributes;publicclassBench{privateSearchValues<char>_searchValues;privatechar[]_textExcept;privateint[]_found;privateDictionary<int,int>_dictionary;privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){_searchValues=SearchValues.Create("ßäöüÄÖÜ");_textExcept=newstring('ß',256).ToCharArray();_textExcept[128]='\n';_found=Enumerable.Range(0,512).ToArray();_dictionary=_found.ToDictionary(k =>k, k =>k);for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicintLastIndexOfAnyExcept()=>_textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);[Benchmark]publicboolContainsKeyTrue_IDictionary()=>ContainsKey(_dictionary);[MethodImpl(MethodImplOptions.NoInlining)]privateboolContainsKey(IDictionary<int,int>collection){boolresult=false;varfound=_found;for(inti=0;i<found.Length;i++)result^=collection.ContainsKey(found[i]);returnresult;}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);}Base is Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 8, 2026
@EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 --filter "ContainsKeyTrueInt32IDictionary" Real perf-repo benchmark this time (random Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 9, 2026
Retry of the @EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 --filter "ContainsKeyTrueInt32IDictionary" Base is Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 9, 2026
EgorBot validation summaryRan the #130045 cases across the hardware classes behind the perf lab, comparing Hardware → lab mapping: Ampere N1 = The two biggest regressions recover to their lab baselines
The PR numbers land right on the pre-#129868 lab baselines. Full matrix (fix effect on the benchmark's Mean; negative = faster)
Reading it
Notes: Note Comment generated with GitHub Copilot CLI. |
Defer the benefit-gate bitvec allocation to when the config is on, correct the "indirection address locals" wording, and soften the config comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…bottom-tested-benefit
Remove the JitLoopInversionRequireBenefitForBottomTested config and always apply the gate: for an already bottom-tested loop with no recognized IV, only invert when the duplicated condition has a call or an indirection whose address has no loop-varying local. Also address review feedback: track GT_LCL_ADDR in indirection addresses via OperIsAnyLocal, and describe the check as an indirection rather than a load. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
AndyAyersMS
commented
Jul 16, 2026
Re-validating the key #130045 cases now that the gate is on by default — the @EgorBot -ubuntu24_azure_ampere -linux_arm64 -ubuntu24_azure_genoa usingSystem;usingSystem.Buffers;usingSystem.Collections.Generic;usingSystem.Linq;usingSystem.Numerics;usingSystem.Runtime.CompilerServices;usingBenchmarkDotNet.Attributes;publicclassBench{privateSearchValues<char>_searchValues;privatechar[]_textExcept;privateint[]_found;privateDictionary<int,int>_dictionary;privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){_searchValues=SearchValues.Create("ßäöüÄÖÜ");_textExcept=newstring('ß',256).ToCharArray();_textExcept[128]='\n';_found=Enumerable.Range(0,512).ToArray();_dictionary=_found.ToDictionary(k =>k, k =>k);for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicintLastIndexOfAnyExcept()=>_textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);[Benchmark]publicboolContainsKeyTrue_IDictionary()=>ContainsKey(_dictionary);[MethodImpl(MethodImplOptions.NoInlining)]privateboolContainsKey(IDictionary<int,int>collection){boolresult=false;varfound=_found;for(inti=0;i<found.Length;i++)result^=collection.ContainsKey(found[i]);returnresult;}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);}Hardware → lab: Ampere N1 = Note Comment generated with GitHub Copilot CLI. |
Workflow state for the Holistic Review Orchestrator. {
"version": 5,
"last_dispatched_commit": "14fabd8de5f4d59307a8eb606fc9edc40c8bbd13",
"last_dispatched_base_ref": "main",
"last_dispatched_base_sha": "6d47555c87f335445a547d00da9cd7a2034d94ce",
"last_reviewed_commit": "14fabd8de5f4d59307a8eb606fc9edc40c8bbd13",
"last_reviewed_base_ref": "main",
"last_reviewed_base_sha": "6d47555c87f335445a547d00da9cd7a2034d94ce",
"last_recorded_worker_run_id": "29685646334",
"review_attempt_commit": "",
"review_attempt_base_ref": "",
"review_attempt_count": 0,
"max_review_attempts": 5,
"review_history_format": "holistic-review-disclosure-v1",
"review_history": [
{
"commit": "14fabd8de5f4d59307a8eb606fc9edc40c8bbd13",
"review_id": 4730705724
}
]
} |
There was a problem hiding this comment.
Holistic Review
Motivation: Fixes #130045, where loop inversion of already bottom-tested loops with no recognized induction variable caused arm64 regressions. Inverting such loops duplicates the exit condition without a clear payoff, so the change gates that specific case on evidence that duplication will actually enable a downstream optimization.
Approach: In optTryInvertWhileLoop, the existing back-edge scan is refactored to track sawExitingCondLatch separately from the early-out IV-test check. When a loop is bottom-tested with no recognized IV (bottomTestedNoIV), a GenTreeVisitor classifies condBlock for a hoisting benefit — a CSE-able call (via optIsCSEcandidate) or an indirection — and records the operand locals. The existing loop-size complexity walk is reused to detect whether any of those locals is stored in the loop (making the candidate loop-variant). Inversion proceeds only when a benefit exists and none of its operands are stored. To support calling the CSE legality filter before tree costs are initialized, CanConsiderTree/optIsCSEcandidate gain a skipCostChecks parameter that bypasses only the cost-based MIN_CSE_COST gate while preserving all legality/structural checks.
Summary: The change is well-scoped, thoroughly commented, and correct. The gate applies only to the narrow bottomTestedNoIV case, and every conservative fallback (size walk skipped when JitLoopInversionSizeLimit < 0, or aborted early on complexity) is biased toward preserving the prior inverting behavior rather than newly suppressing it. The assert(analyzedIteration) holds because sawExitingCondLatch is only set inside branches that first call isIvTest, which forces AnalyzeIteration. The skipCostChecks refactor is a clean, side-effect-free restructuring of the cost block. Piggy-backing local-store detection on the existing complexity walk keeps the added cost negligible. I have only minor, non-blocking observations noted below. LGTM.
Detailed Findings
Non-blocking observations (not actionable, no changes required):
optcse.cpp— The indirection branch inCondClassifier::PreOrderVisitsets*m_hasCandidate = truefor anyOperIsIndirnode without excluding volatile indirections, which LICM cannot hoist. Because the address-local store check still runs, a genuinely loop-variant indirection is filtered out, but a volatile invariant load would be treated as a benefit. This only makes the gate slightly more permissive (retaining an inversion that may not pay off), so it does not regress correctness or the targeted #130045 scenario. Optional tightening if the heuristic proves too permissive in practice.optcse.cpp—optGetCSEheuristic()now constructs and caches the CSE heuristic during loop inversion (earlier than the CSE phase) for methods that reach the classifier with a call incondBlock. This is harmless — the object is cached inoptCSEheuristicand reused unchanged byoptOptimizeValnumCSEs— but it is a phase-ordering behavior change worth noting; the heuristic is now instantiated for some methods that previously deferred it until CSE. No functional impact.
Given the heuristic nature of the change, the arm64-focused CI/perf validation referenced in the PR is the right gate for confirming the regression fix and absence of broad CQ loss; the code itself is sound.
Note
This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.
Generated by Holistic Review · 118.6 AIC · ⌖ 14.9 AIC · ⊞ 10K
AndyAyersMS
commented
Jul 21, 2026
@EgorBot -ubuntu24_azure_ampere -linux_arm64 -ubuntu24_azure_genoa usingSystem;usingSystem.Buffers;usingSystem.Collections.Generic;usingSystem.Linq;usingSystem.Numerics;usingSystem.Runtime.CompilerServices;usingBenchmarkDotNet.Attributes;publicclassBench{privateSearchValues<char>_searchValues;privatechar[]_textExcept;privateint[]_found;privateDictionary<int,int>_dictionary;privatereadonlyint[]_a=newint[512];privatereadonlyint[]_b=newint[512];privateBigInteger_x,_y;[GlobalSetup]publicvoidSetup(){_searchValues=SearchValues.Create("ßäöüÄÖÜ");_textExcept=newstring('ß',256).ToCharArray();_textExcept[128]='\n';_found=Enumerable.Range(0,512).ToArray();_dictionary=_found.ToDictionary(k =>k, k =>k);for(inti=0;i<_a.Length;i++){_a[i]=i;_b[i]=i;}byte[]bytes=newbyte[259];newRandom(42).NextBytes(bytes);bytes[^1]&=0x7f;_x=newBigInteger(bytes);_y=newBigInteger(bytes);}[Benchmark]publicintLastIndexOfAnyExcept()=>_textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);[Benchmark]publicboolContainsKeyTrue_IDictionary()=>ContainsKey(_dictionary);[MethodImpl(MethodImplOptions.NoInlining)]privateboolContainsKey(IDictionary<int,int>collection){boolresult=false;varfound=_found;for(inti=0;i<found.Length;i++)result^=collection.ContainsKey(found[i]);returnresult;}[Benchmark]publicboolSpan_SequenceEqual()=>_a.AsSpan().SequenceEqual(_b);[Benchmark]publicintBigInteger_CompareTo()=>_x.CompareTo(_y);} |
AndyAyersMS
commented
Jul 21, 2026
Checking whether this PR also addresses #130046 (Windows Zen4). The fleet has no Windows Zen4, so best-available Zen4 is Linux Genoa; Windows here is Turin (Zen5) as an anchor. @EgorBot -ubuntu24_azure_genoa -windows_x64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 usingSystem;usingSystem.Collections.Generic;usingSystem.Globalization;usingSystem.Linq;usingBenchmarkDotNet.Attributes;publicclassBench{privateSortedDictionary<int,int>_sortedDict;privateDictionary<string,string>_stringDict;privateCompareInfo_en;privatestring_text;[GlobalSetup]publicvoidSetup(){vard=Enumerable.Range(0,512).ToDictionary(k =>k, k =>k);_sortedDict=newSortedDictionary<int,int>(d);varrnd=newRandom(12345);varkeys=newHashSet<string>();while(keys.Count<512)keys.Add(rnd.Next().ToString("X8")+rnd.Next().ToString("X8"));_stringDict=keys.ToDictionary(k =>k, k =>k);_en=CultureInfo.GetCultureInfo("en-US").CompareInfo;_text="NET Conf provides a wide selection of live sessions streaming here that feature speakers from the community and .NET product teams. It is a chance to learn, ask questions live, and get inspired for your next software project";}// System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512)[Benchmark]publicSortedDictionary<int,int>SortedDictionaryDeepCopy_Int()=>newSortedDictionary<int,int>(_sortedDict);// System.Collections.CtorFromCollection<String>.Dictionary(Size: 512)[Benchmark]publicDictionary<string,string>Dictionary_String()=>newDictionary<string,string>(_stringDict);// System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(en-US, OrdinalIgnoreCase, false)[Benchmark]publicintStringSearch_LastIndexOf_Word_NotFound()=>_en.LastIndexOf(_text,"word",CompareOptions.OrdinalIgnoreCase);}Base is Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 22, 2026
Re-running (previous attempt hit an EgorBot arg-parsing glitch on @EgorBot -ubuntu24_azure_genoa usingSystem;usingSystem.Collections.Generic;usingSystem.Globalization;usingSystem.Linq;usingBenchmarkDotNet.Attributes;publicclassBench{privateSortedDictionary<int,int>_sortedDict;privateDictionary<string,string>_stringDict;privateCompareInfo_en;privatestring_text;[GlobalSetup]publicvoidSetup(){vard=Enumerable.Range(0,512).ToDictionary(k =>k, k =>k);_sortedDict=newSortedDictionary<int,int>(d);varrnd=newRandom(12345);varkeys=newHashSet<string>();while(keys.Count<512)keys.Add(rnd.Next().ToString("X8")+rnd.Next().ToString("X8"));_stringDict=keys.ToDictionary(k =>k, k =>k);_en=CultureInfo.GetCultureInfo("en-US").CompareInfo;_text="NET Conf provides a wide selection of live sessions streaming here that feature speakers from the community and .NET product teams. It is a chance to learn, ask questions live, and get inspired for your next software project";}[Benchmark]publicSortedDictionary<int,int>SortedDictionaryDeepCopy_Int()=>newSortedDictionary<int,int>(_sortedDict);[Benchmark]publicDictionary<string,string>Dictionary_String()=>newDictionary<string,string>(_stringDict);[Benchmark]publicintStringSearch_LastIndexOf_Word_NotFound()=>_en.LastIndexOf(_text,"word",CompareOptions.OrdinalIgnoreCase);}Genoa Zen4 is the closest available mapping for #130046's Windows Zen4. Note Comment generated with GitHub Copilot CLI. |
AndyAyersMS
commented
Jul 22, 2026
@EgorBot -ubuntu24_azure_genoa usingSystem;usingSystem.Collections.Generic;usingSystem.Globalization;usingSystem.Linq;usingBenchmarkDotNet.Attributes;publicclassBench{privateSortedDictionary<int,int>_sortedDict;privateDictionary<string,string>_stringDict;privateCompareInfo_en;privatestring_text;[GlobalSetup]publicvoidSetup(){vard=Enumerable.Range(0,512).ToDictionary(k =>k, k =>k);_sortedDict=newSortedDictionary<int,int>(d);varrnd=newRandom(12345);varkeys=newHashSet<string>();while(keys.Count<512)keys.Add(rnd.Next().ToString("X8")+rnd.Next().ToString("X8"));_stringDict=keys.ToDictionary(k =>k, k =>k);_en=CultureInfo.GetCultureInfo("en-US").CompareInfo;_text="NET Conf provides a wide selection of live sessions streaming here that feature speakers from the community and .NET product teams. It is a chance to learn, ask questions live, and get inspired for your next software project";}[Benchmark]publicSortedDictionary<int,int>SortedDictionaryDeepCopy_Int()=>newSortedDictionary<int,int>(_sortedDict);[Benchmark]publicDictionary<string,string>Dictionary_String()=>newDictionary<string,string>(_stringDict);[Benchmark]publicintStringSearch_LastIndexOf_Word_NotFound()=>_en.LastIndexOf(_text,"word",CompareOptions.OrdinalIgnoreCase);}Third attempt — checking whether this PR also addresses the #130046 (Windows Zen4) regressions. Genoa Zen4 is the closest available mapping in the fleet. Gate is on by default on this branch, so no env var needed. (Previous two runs hit an EgorBot parser bug that injects a stray Note Comment generated with GitHub Copilot CLI. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (2)
src/coreclr/jit/optimizer.cpp:2190
- The comment and JITDUMP message claim the heuristic found a "loop-invariant hoisting candidate" and that "LICM can lift" it. Here the heuristic only checks for a call passing the CSE legality filter or an indirection whose operands have no loop-varying locals (plus a local-store check), which is not the same as full loop-invariance/hoistability (aliasing, indirect writes, etc.). Reword to match the implemented heuristic to avoid over-claiming.
// Skip the inversion unless the duplicated test carries a benefit: a loop-invariant hoisting
// candidate (a CSE-able call or an indirection whose operands have no loop-varying local), which
// LICM can lift once the body dominates the back-edge. condCandidateStored is left conservatively
// false if the size walk above was skipped or aborted early, keeping the inversion.
if (bottomTestedNoIV)
src/coreclr/jit/optimizer.cpp:2063
CondClassifier::PreOrderVisitcallsCollectLocals(...)(which recursively walks the node's operands) but then returnsWALK_CONTINUE, so theGenTreeVisitorwill also walk the same subtree. This duplicates work during loop inversion and can become noticeable on large conditions/methods. ReturnWALK_SKIP_SUBTREESafterCollectLocals(...)to avoid the double traversal.
{
*m_hasCandidate = true;
CollectLocals(n);
}
return WALK_CONTINUE;
AndyAyersMS
commented
Aug 10, 2026
@jakobbotsch maybe we should consider this? |
| if (n->IsCall()) | ||
| { | ||
| // Only a call the CSE heuristic would consider (no persistent side effects, not an | ||
| // allocator) is a reuse candidate; LICM/CSE cannot lift a call with side effects, | ||
| // so inverting for it buys nothing. Its arguments must also be loop-invariant. Skip | ||
| // the cost-based checks: tree costs are not initialized this early. | ||
| if (m_compiler->optIsCSEcandidate(n, /* isReturn */ false, /* skipCostChecks */ true)) | ||
| { | ||
| *m_hasCandidate = true; | ||
| CollectLocals(n); | ||
| } | ||
| return WALK_CONTINUE; | ||
| } | ||
| if (n->OperIsIndir()) | ||
| { | ||
| *m_hasCandidate = true; | ||
| CollectLocals(n->AsIndir()->Addr()); | ||
| } |
There was a problem hiding this comment.
Are these cases really common? What are some example loops we invert where such a call/indir is a deciding factor?
There was a problem hiding this comment.
Indir is common, call is not. So I've trimmed it down.
Common indir examples are fields in unpromoted enumerators (eg in the dictionary IterateForEach benchmark -- there the MoveNext is too big to inline, so the enumerator escapes), or the scan pointers in sorting.
If we don't invert we end up not being able to host these and we have indirs per loop iteration that slow down perf.
Calls in the duplicated loop condition are almost never CSE-able, so the call clause never fires: over a PMI run of all framework assemblies (96,440 methods) the bottom-tested/no-IV gate keeps 1,524 inversions and every one is decided by an indirection, not a call. Restrict the benefit test to loop-invariant indirections and revert the now-unused CSE cost-check bypass. No asm diffs vs the prior JIT across the collection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 931deb5a-5ece-4a78-ba06-49c40d245e8b
…bottom-tested-benefit
…-benefit' into jit-loop-inversion-bottom-tested-benefit
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (4)
src/coreclr/jit/optimizer.cpp:2139
- This inline comment still says a store to one of the recorded locals makes the candidate “not hoistable”. That’s stronger than what this check can prove; at most it’s a conservative signal the indirection address may vary within the loop.
// Note whether any local appearing in a condition hoisting candidate's operands is
// stored in the loop (making that candidate loop-variant, hence not hoistable).
if (bottomTestedNoIV && !condCandidateStored && tree->OperIsLocalStore() &&
BitVecOps::IsMember(&condTraits, condCandidateLocals, tree->AsLclVarCommon()->GetLclNum()))
{
src/coreclr/jit/optimizer.cpp:2176
- The skip comment and JITDUMP message both still describe this as requiring a “loop-invariant hoisting candidate”. That phrasing over-claims what the heuristic checks (and becomes inaccurate once shared helper/array length are considered as benefits). Reword these to match the actual gate conditions.
// Skip the inversion unless the duplicated test carries a benefit: a loop-invariant hoisting
// candidate (an indirection whose address has no loop-varying local), which LICM can lift once the
// body dominates the back-edge. condCandidateStored is left conservatively false if the size walk
// above was skipped or aborted early, keeping the inversion.
if (bottomTestedNoIV)
src/coreclr/jit/optimizer.cpp:2049
CondClassifier::PreOrderVisitonly treatsOperIsIndir()as a “benefit” signal. The PR intent/description mentions calls as well, and this method already has an existing notion of benefit candidates (Compiler::IsSharedStaticHelper/OperIsArrLength) inoptInvertCountTreeInfo. Also,CollectLocalsrecursively walks the indirection address but the visitor then walks the same subtree again, which is redundant compile-time work; returningWALK_SKIP_SUBTREESafterCollectLocalsavoids the double-walk.
if (n->OperIsIndir())
{
*m_hasCandidate = true;
CollectLocals(n->AsIndir()->Addr());
}
src/coreclr/jit/optimizer.cpp:1997
- The new explanatory comment currently claims the indirection is “loop-invariant” and that LICM “can hoist” it. But the heuristic only tracks whether locals appear in the address and whether those locals are directly stored in the loop; it doesn’t prove invariance or hoistability (aliasing/side effects/etc.). The comment should be softened to describe what’s actually checked.
This issue also appears in the following locations of the same file:
- line 2135
- line 2172
// If the loop is already bottom-tested (has an exiting BBJ_COND latch that is not the IV test)
// and no induction variable was recognized, only invert when the block that would be duplicated
// (condBlock) contains a loop-invariant hoisting candidate: an indirection whose address has no
// loop-varying local. Once the body dominates the back-edge such a load can be hoisted by LICM,
// which is the benefit that makes bottom-testing worthwhile. Classify condBlock here and record
Make loop inversion a bit less aggressive.
If a loop is bottom-tested and has no evident IV, only invert if there is a hint that inversion might lead to a beneficial CSE (call or possibly invariant load). Analysis is a approximate and piggy backs on existing IR walks we already do.
Fixes#130045.
Note
This change and PR description were produced with GitHub Copilot CLI.