[JIT] JIT regression in .NET 10: foreach over Span<T> field in ref struct triggers excessive inlining of ArrayPool internals

[ablanchet]

Description

The .NET 10 compiler seems to have a performance regression when using foreach on a Span<T> field stored in a ref struct that is backed by ArrayPool<T>. The JIT produces code that is 2x slower than .NET 9 for the same pattern. The equivalent for loop is unaffected.

Here is the benchmark that repro the issue:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using System;
using System.Buffers;
using System.Runtime.InteropServices;

namespace RentSpanUnmanagedPerfTests;

[SimpleJob(RuntimeMoniker.Net90)]
[SimpleJob(RuntimeMoniker.Net10_0)]
[DisassemblyDiagnoser(maxDepth: 3, exportDiff: true)]
public class ArrayPoolForeachRepro
{
 private const int Size = 1000;

 [Benchmark(Description = "ArrayPool + Span - foreach (SLOW on .NET 10 and .NET 9)")]
 public int ArrayPool_Foreach()
 {
 var pool = ArrayPool<int>.Shared;
 var array = pool.Rent(Size);
 try
 {
 var span = new Span<int>(array, 0, Size);

 // Write
 for (var i = 0; i < Size; i++)
 {
 span[i] = i * 2;
 }

 // Read with foreach
 var sum = 0;
 foreach (ref readonly var value in span)
 {
 sum += value;
 }

 return sum;
 }
 finally
 {
 pool.Return(array);
 }
 }

 [Benchmark(Description = "ArrayPool + Span - for loop (OK on .NET 10 and .NET 9)")]
 public int ArrayPool_ForLoop()
 {
 var pool = ArrayPool<int>.Shared;
 var array = pool.Rent(Size);
 try
 {
 var span = new Span<int>(array, 0, Size);

 // Write
 for (var i = 0; i < Size; i++)
 {
 span[i] = i * 2;
 }

 // Read with for loop
 var sum = 0;
 for (var i = 0; i < span.Length; i++)
 {
 sum += span[i];
 }

 return sum;
 }
 finally
 {
 pool.Return(array);
 }
 }

 [Benchmark(Description = "RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)")]
 public int RefStruct_Foreach()
 {
 using var wrapper = new SpanWrapper(Size);

 // Write
 for (var i = 0; i < Size; i++)
 {
 wrapper.Span[i] = i * 2;
 }

 // Read with foreach
 var sum = 0;
 var span = wrapper.Span;
 foreach (ref readonly var value in span)
 {
 sum += value;
 }

 return sum;
 }

 [Benchmark(Description = "RefStruct + Span field - for loop (OK on .NET 10 and .NET 9)")]
 public int RefStruct_ForLoop()
 {
 using var wrapper = new SpanWrapper(Size);

 // Write
 for (var i = 0; i < Size; i++)
 {
 wrapper.Span[i] = i * 2;
 }

 // Read with for loop
 var sum = 0;
 for (var i = 0; i < wrapper.Span.Length; i++)
 {
 sum += wrapper.Span[i];
 }

 return sum;
 }

 private readonly ref struct SpanWrapper
 {
 private readonly int[] m_Array;
 public readonly Span<int> Span;

 public SpanWrapper(int length)
 {
 m_Array = ArrayPool<int>.Shared.Rent(length);
 Span = new Span<int>(m_Array, 0, length);
 }

 public void Dispose()
 {
 ArrayPool<int>.Shared.Return(m_Array);
 }
 }
}

On .NET 9, the test RefStruct_Foreach runs in ~653 ns. On .NET 10, it runs in ~1,353 ns — a 2.07x regression.

The disassembly shows that on .NET 10, the JIT aggressively inlines the entire SharedArrayPool<T>.Rent() method (bucket management, thread-static access, hash code computation, monitor operations) into the method body when ƒoreach is used on the Span field. On .NET 9, Rent and return are kept as normal calls, producing a compact method body of ~198 bytes with clean loops.

Disassembly diff is here: https://gist.github.com/ablanchet/8831824c3b81134fef9d5e1d701633ff

Configuration

• Runtimes:
  • .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4 (Job: .NET 10.0(Runtime=.NET 10.0))
  • .NET 9.0.11 (9.0.11, 9.0.1125.51716), X64 RyuJIT x86-64-v4 (Job: .NET 9.0(Runtime=.NET 9.0))
• OS:
  • Windows 11 for disassembly export.
  • Similar results on MacOS 26.2

Regression?

Yes, same exact code is running 2x slower on .NET 10 compared to .NET 9.

Data

Here are the results of the benchmarkdotnet run

| Method | Job | Runtime | Mean | Error | StdDev | Median | Code Size |
|--------------------------------------------------------------------- |---------- |---------- |-----------:|---------:|---------:|-----------:|----------:|
| 'ArrayPool + Span - foreach (SLOW on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 1,340.5 ns | 26.83 ns | 71.16 ns | 1,335.5 ns | 2,589 B |
| 'ArrayPool + Span - for loop (OK on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 673.3 ns | 13.50 ns | 30.19 ns | 686.8 ns | 2,573 B |
| 'RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)' | .NET 10.0 | .NET 10.0 | 1,353.4 ns | 26.92 ns | 75.49 ns | 1,364.8 ns | 2,547 B |
| 'RefStruct + Span field - for loop (OK on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 659.4 ns | 13.14 ns | 33.21 ns | 670.4 ns | 2,535 B |
| 'ArrayPool + Span - foreach (SLOW on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 1,394.6 ns | 27.74 ns | 62.05 ns | 1,406.9 ns | 2,008 B |
| 'ArrayPool + Span - for loop (OK on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 655.0 ns | 13.03 ns | 32.92 ns | 666.8 ns | 2,004 B |
| 'RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)' | .NET 9.0 | .NET 9.0 | 652.8 ns | 13.62 ns | 40.15 ns | 655.5 ns | 2,110 B |
| 'RefStruct + Span field - for loop (OK on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 666.7 ns | 13.36 ns | 30.16 ns | 679.8 ns | 2,110 B |

Analysis

Looking at the disassembly diff it looks like in .NET the foreach is inlining too much code compared to .NET 9, but that's probably a symptom but not the root cause.

[hez2010]

I guess it's because inlining the late-devirted method (SharedArrayPool<T>.Rent) running out of inliner's budget, causing more important methods not being inlined anymore.
A simple solution is to put a MethodImplOptions.NoInling on SharedArrayPool<T>.Rent().

[JulieLeeMSFT]

A simple solution is to put a MethodImplOptions.NoInling on SharedArrayPool<T>.Rent().

@ablanchet, please try hez's suggestion if the regression goes away.

[hez2010]

A simple solution is to put a MethodImplOptions.NoInling on SharedArrayPool<T>.Rent().

@ablanchet, please try hez's suggestion if the regression goes away.

This is not actionable on user's side as SharedArrayPool<T> is not user code.

A workaround is adding a wrapper function on ArrayPool like

[MethodImpl(MethodImplOptions.NoInlining)]
T[] RentFromSharedArrayPool(int size) => ArrayPool<T>.Shared.Rent(size);

and use it instead of using ArrayPool<T>.Shared.Rent directly

[ablanchet]

@JulieLeeMSFT @hez2010

After Hez's message I did try to add the no-inline attribute on SharedArrayPool's Rent definition. Here's what I did to make the change, build the runtime, librairies, test harness, and run the perf tests again:

• cloned the runtime repo and checked-out the release/10.0 branch
• made sure the requirements were installed on my machine
• made the code change in runtime/src/libraries/System.Private.CoreLib/src/System/Buffers/SharedArrayPool.cs
  • added [MethodImpl(MethodImplOptions.NoInlining)] line 50 on Rent()
• built the runtime and libraries in release with ./build.sh -subset clr+libs -configuration Release
• built the test environment in release with ./src/tests/build.sh -release -generatelayoutonly
• re run my test with

dotnet run -c Release -f net9.0 \
 --coreRun /opt/Work/runtime/artifacts/bin/testhost/net10.0-osx-Release-arm64/shared/Microsoft.NETCore.App/10.0.4/corerun

In the end I got

| Method | Mean | Error | StdDev |
|--------------------------------------------------------------------- |-----------:|---------:|----------:|
| 'RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)' | 1,920.4 ns | 74.70 ns | 220.27 ns |

---

Then I changed my SpanWrapper test struct following Hez's suggestion like this:

private readonly ref struct SpanWrapper
{
 private readonly int[] m_Array;
 public readonly Span<int> Span;
 public SpanWrapper(int length)
 {
 m_Array = RentFromSharedArrayPool<int>(length);
 Span = new Span<int>(m_Array, 0, length);
 }
 
 [MethodImpl(MethodImplOptions.NoInlining)]
 T[] RentFromSharedArrayPool<T>(int size) => ArrayPool<T>.Shared.Rent(size);
 public void Dispose()
 {
 ArrayPool<int>.Shared.Return(m_Array);
 }
}

And re-run my original test (on .NET 9 and .NET 10 installed binaries - not the custom built one) and got this:

| Method | Job | Runtime | Mean | Error | StdDev | Median |
|--------------------------------------------------------------------- |---------- |---------- |-----------:|---------:|----------:|-----------:|
| 'ArrayPool + Span - foreach (SLOW on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 1,927.1 ns | 63.06 ns | 185.92 ns | 1,956.7 ns |
| 'ArrayPool + Span - for loop (OK on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 541.4 ns | 0.68 ns | 0.64 ns | 541.5 ns |
| 'RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)' | .NET 10.0 | .NET 10.0 | 2,021.5 ns | 52.03 ns | 153.40 ns | 2,056.9 ns |
| 'RefStruct + Span field - for loop (OK on .NET 10 and .NET 9)' | .NET 10.0 | .NET 10.0 | 533.9 ns | 0.38 ns | 0.30 ns | 534.0 ns |
| 'ArrayPool + Span - foreach (SLOW on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 2,073.0 ns | 48.75 ns | 143.74 ns | 2,160.0 ns |
| 'ArrayPool + Span - for loop (OK on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 525.8 ns | 0.54 ns | 0.50 ns | 525.7 ns |
| 'RefStruct + Span field - foreach (SLOW on .NET 10, FAST on .NET 9)' | .NET 9.0 | .NET 9.0 | 542.4 ns | 3.66 ns | 3.42 ns | 542.3 ns |
| 'RefStruct + Span field - for loop (OK on .NET 10 and .NET 9)' | .NET 9.0 | .NET 9.0 | 541.5 ns | 3.06 ns | 2.86 ns | 540.9 ns | 

---

Conclusion:

For some reason I don't have the same low and high values in performance tests, probably due to background activity on my machine.
Nonetheless these results are still useful IMO since they are unfortunately showing no improvements by adding [MethodImpl(MethodImplOptions.NoInlining)] on either a user defined wrapper function on Rent or on Rent directly.

---

Hope this helps and let me know if you need anything else.

[JulieLeeMSFT]

cc @jakobbotsch.

[AndyAyersMS]

Inner loop codegen for RefStruct_Foreach.

Looks like the relevant fields didn't get promoted:

;; .NET 9
 align [1 bytes for IG06]
 
G_M000_IG06: ;; offset=0x0060
 add esi, dword ptr [rcx+rdx]
 add rdx, 4
 dec r8d
 jne SHORT G_M000_IG06

;; .NET 10

 align [4 bytes for IG08]

G_M000_IG08: ;; offset=0x00E0
 mov eax, dword ptr [rbp-0x3C]
 add ecx, dword ptr [r14+4*rax]
 mov eax, dword ptr [rbp-0x3C]
 inc eax
 mov dword ptr [rbp-0x3C], eax
 cmp dword ptr [rbp-0x3C], 0x3E8
 jl SHORT G_M000_IG08

Seems to be related to EH liveness... in a checked .NET 10 jit we see

Local V05 should not be enregistered because: live in/out of a handler

G_M61959_IG08: ; offs=0x0000E0, size=0x0018, bbWeight=982.32, PerfScore 9086.51, gcrefRegs=0000 {}, byrefRegs=4000 {r14}, loop=IG08, BB06 [0005], byref, isz

IN002c: 0000E0 mov eax, dword ptr [V05 rbp-0x3C]
IN002d: 0000E3 add ecx, dword ptr [r14+4*rax]
IN002e: 0000E7 mov eax, dword ptr [V05 rbp-0x3C]
IN002f: 0000EA inc eax
IN0030: 0000EC mov dword ptr [V05 rbp-0x3C], eax
IN0031: 0000EF cmp dword ptr [V05 rbp-0x3C], 0x3E8
IN0032: 0000F6 jl SHORT G_M61959_IG08

which does not happen in the .NET 9 jit.

We indeed do a lot of (probably pointless) inlining in .NET 10 but the EH is there in the root method (as is V05 as a local). A cursory look shows that during SSA building V05 is now live everywhere, which seems odd:

*************** Starting PHASE Build SSA representation
*************** In SsaBuilder::Build()
....

** Memory liveness computed, GcHeap states and ByrefExposed states match
*************** In fgInterBlockLocalVarLiveness()

BB liveness after fgLiveVarAnalysis():

BB01 IN (1)={V05 } + ByrefExposed + GcHeap
 OUT(6)={V05 V15 V08 V12 V13 V14} + ByrefExposed + GcHeap

@jakobbotsch I'll let you dig in from here...