Uh oh!
There was an error while loading. Please reload this page.
JIT: Fix uninitialized gtCallMethHnd for indirect calls causing false recursive tail call detection - #127293
Conversation
… recursive tail call detection PR dotnet#125211 broke the union between gtCallMethHnd and gtCallAddr into separate fields but did not initialize gtCallMethHnd for CT_INDIRECT calls. Since the arena allocator does not zero memory in Release builds, gtCallMethHnd contains stale data that can accidentally match the compiled method's handle, causing gtIsRecursiveCall() to return true. Combined with the CORINFO_VIRTUALCALL_LDVIRTFTN import path (used for generic interface calls in R2R) which creates CT_INDIRECT calls without GTF_CALL_VIRT_* flags, the recursive tail call optimization incorrectly transforms the call into a backward jump, creating an infinite allocating loop. Fix by: 1. Initializing gtCallMethHnd = NO_METHOD_HANDLE in gtNewCallNode for CT_INDIRECT 2. Adding a defense-in-depth check in gtIsRecursiveCall to return false for indirect calls 3. Initializing gtCallMethHnd = NO_METHOD_HANDLE in gtCloneExprCallHelper for CT_INDIRECT Fixesdotnet#126930 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
There was a problem hiding this comment.
Pull request overview
This PR fixes a CoreCLR JIT correctness issue introduced when GenTreeCall::gtCallMethHnd was split from the former gtCallAddr union: indirect call nodes (CT_INDIRECT) could leave gtCallMethHnd uninitialized, leading to nondeterministic false “recursive tail call” detection and potential infinite loops/OOMs.
Changes:
- Initialize
gtCallMethHndtoNO_METHOD_HANDLEfor newly createdCT_INDIRECTcalls. - Initialize
gtCallMethHndtoNO_METHOD_HANDLEwhen cloningCT_INDIRECTcalls. - Add a guard in
gtIsRecursiveCall(GenTreeCall*)to always returnfalseforCT_INDIRECTcalls (defense-in-depth).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/coreclr/jit/gentree.cpp | Ensures indirect call creation/cloning does not leave gtCallMethHnd holding stale arena data. |
src/coreclr/jit/compiler.h | Prevents recursive-call checks from consulting method handles on indirect calls. |
Uh oh!
There was an error while loading. Please reload this page.
AndyAyersMS
commented
Apr 24, 2026
@dotnet/jit-contrib PTAL Suggest we take this to fix the ci-optional issue and then circle back to add better protection. |
Uh oh!
There was an error while loading. Please reload this page.
Note
This PR was generated with the assistance of GitHub Copilot.
Fixes#126930
Problem
PR #125211 removed the union between
gtCallMethHndandgtCallAddrinGenTreeCall, making them separate fields. However,gtNewCallNodewas not updated to initializegtCallMethHndforCT_INDIRECTcalls. Since the JIT arena allocator doesn't zero memory in Release builds,gtCallMethHndcontains stale arena data.When the stale data coincidentally matches the compiled method's handle,
gtIsRecursiveCall()falsely returns true for indirect calls. Combined with!call->IsVirtual()being true forCORINFO_VIRTUALCALL_LDVIRTFTNcalls (which lackGTF_CALL_VIRT_*flags),fgMorphRecursiveFastTailCallIntoLoopincorrectly transforms the call into a backward jump — creating an infinite allocating loop.This only manifests non-deterministically depending on arena memory layout, which is why it appears on x64 CI but not on arm64 or in local builds.
Fix
Three surgical changes:
gentree.cpp(gtNewCallNode): InitializegtCallMethHnd = NO_METHOD_HANDLEforCT_INDIRECTcalls — the primary fix that prevents stale data.compiler.h(gtIsRecursiveCall): Defense-in-depth — returnfalseforCT_INDIRECTcalls since indirect calls can never be recursive by definition.gentree.cpp(gtCloneExprCallHelper): Same initialization for cloned indirect calls to prevent the same issue in clone paths.