[Perf] Windows/arm64: 28 Regressions on 7/18/2023 1:13:00 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetBytes - Duration of single invocation 📝 - Benchmark Source	134.64 μs	163.41 μs	1.21	0.03	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	144.34 μs	190.21 μs	1.32	0.01	False
GetString - Duration of single invocation 📝 - Benchmark Source	319.91 μs	361.24 μs	1.13	0.00	False
GetString - Duration of single invocation 📝 - Benchmark Source	200.01 μs	236.03 μs	1.18	0.00	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	159.97 μs	208.18 μs	1.30	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	187.68 μs	251.38 μs	1.34	0.05	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	203.39 μs	238.89 μs	1.17	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	44.30 μs	135.22 μs	3.05	0.01	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	42.38 μs	78.88 μs	1.86	0.04	False
GetString - Duration of single invocation 📝 - Benchmark Source	261.72 μs	306.76 μs	1.17	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Details

Payloads

Baseline
Compare

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.41458333333333 > 141.77467636148876.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -156.9943601397984 (T) = (0 -163313.67105263157) / Math.Sqrt((332399.08210140927 / (15)) + (20366.576792853983 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.20901931770273424 = (135079.45544074927 - 163313.67105263157) / 135079.45544074927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.21237931910568 > 151.46738830222137.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -57.93111328415174 (T) = (0 -191002.3032508227) / Math.Sqrt((264013.2822441723 / (15)) + (1247959.6357985316 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.32089916063131374 = (144600.21547711076 - 191002.3032508227) / 144600.21547711076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 361.24470930232553 > 335.2484196428572.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/18/2023 4:36:42 PM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -14.118172520751896 (T) = (0 -364439.96422182466) / Math.Sqrt((156432.7000792867 / (15)) + (20419308.001166407 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.14135101329335703 = (319305.770071765 - 364439.96422182466) / 319305.770071765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.02786796536802 > 209.01690668706294.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -31.45956270973919 (T) = (0 -237201.51176948054) / Math.Sqrt((948388.27929041 / (15)) + (2754879.557863424 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.18932988111238652 = (199441.31189878515 - 237201.51176948054) / 199441.31189878515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.17916666666665 > 167.74164674908423.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -105.10239953388513 (T) = (0 -207929.19060106558) / Math.Sqrt((2181557.5854358426 / (15)) + (124976.06674677305 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.2995270615430891 = (160003.7404024242 - 207929.19060106558) / 160003.7404024242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 251.37670546558707 > 199.26802149662032.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -49.931345545245065 (T) = (0 -252501.08962795482) / Math.Sqrt((3578319.5423276178 / (15)) + (2528479.4891668903 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.31998419269850836 = (191290.99501695886 - 252501.08962795482) / 191290.99501695886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 238.88602810650892 > 213.3333234577922.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/14/2023 8:42:09 PM, 3/21/2023 2:53:21 PM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -46.44101491427244 (T) = (0 -238147.19893087685) / Math.Sqrt((337750.0174463117 / (15)) + (1091737.1015303673 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.17236098584785833 = (203134.70151742332 - 238147.19893087685) / 203134.70151742332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.2222883064516 > 46.67656657750623.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -10.519272418769509 (T) = (0 -144750.35992703532) / Math.Sqrt((105388.31582920565 / (15)) + (181568297.61394644 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.2514945147506236 = (44518.10060584927 - 144750.35992703532) / 44518.10060584927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.88215829439251 > 44.312702125.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -20.02921160723858 (T) = (0 -77139.65478125423) / Math.Sqrt((224515.86856747864 / (15)) + (6072636.986598566 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8300022207196299 = (42152.765667639374 - 77139.65478125423) / 42152.765667639374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.76269403594773 > 275.3442643065268.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -108.94988157600964 (T) = (0 -306379.07110760966) / Math.Sqrt((241114.1247489303 / (15)) + (294333.10229330295 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.16778130939409266 = (262359.9715485903 - 306379.07110760966) / 262359.9715485903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToString_MultipleSegments - Duration of single invocation 📝 - Benchmark Source	21.13 μs	96.64 μs	4.57	0.07	False
ToString_SingleSegment - Duration of single invocation 📝 - Benchmark Source	18.98 μs	79.88 μs	4.21	0.02	False
ctor_capacity - Duration of single invocation 📝 - Benchmark Source	5.52 μs	10.16 μs	1.84	0.05	False
Append_Char_Capacity - Duration of single invocation 📝 - Benchmark Source	284.55 μs	420.04 μs	1.48	0.48	False
ctor_string - Duration of single invocation 📝 - Benchmark Source	14.37 μs	72.14 μs	5.02	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Details

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.64492912371134 > 22.19193447327905.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -8.8614939204825 (T) = (0 -89009.10485155522) / Math.Sqrt((301034.7288344912 / (15)) + (116611624.63049658 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.172435378856418 = (21332.650303610182 - 89009.10485155522) / 21332.650303610182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.88275290697675 > 20.413894121914147.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.62250119071027 (T) = (0 -77905.38003955394) / Math.Sqrt((50497.475381388256 / (15)) + (7820006.9136397075 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.03541094841131 = (19305.439033470506 - 77905.38003955394) / 19305.439033470506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.163017519393357 > 5.795809405674804.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -135.76333695931683 (T) = (0 -10131.36906001631) / Math.Sqrt((2180.440353936697 / (15)) + (2003.2499618812392 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8309336596506445 = (5533.444101928548 - 10131.36906001631) / 5533.444101928548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 420.04497722672073 > 300.31783359375.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/30/2023 5:15:39 PM, 5/18/2023 7:10:13 PM, 6/1/2023 5:52:46 AM, 6/28/2023 11:43:34 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -18.9499134774535 (T) = (0 -420434.1379859472) / Math.Sqrt((781893432.2642138 / (15)) + (302892.19304345635 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.48343140528585415 = (283420.00613431155 - 420434.1379859472) / 283420.00613431155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.13512505671508 > 15.111548469023417.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -71.20252621282843 (T) = (0 -72955.43077953125) / Math.Sqrt((41195.66829324731 / (15)) + (1345802.9577699201 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -4.056335134774942 = (14428.51963624411 - 72955.43077953125) / 14428.51963624411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Diagnostics.Perf_Process

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetProcesses - Duration of single invocation 📝 - Benchmark Source	1.07 ms	2.02 ms	1.89	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Process*'

Details

Payloads

Baseline
Compare

System.Diagnostics.Perf_Process.GetProcesses

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0161652529761906 > 1.1468048437500005.
IsChangePoint: Marked as a change because one of 4/8/2023 9:16:19 PM, 4/27/2023 1:05:59 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -11.027556022158558 (T) = (0 -1960596.8091804031) / Math.Sqrt((30159161263.010315 / (15)) + (6175703891.771193 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.6711666399223366 = (1173190.4900109286 - 1960596.8091804031) / 1173190.4900109286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AllocateUninitializedArray - Duration of single invocation	384.51 ns	1.72 μs	4.47	0.09	False
AllocateArray - Duration of single invocation	775.27 ns	1.88 μs	2.42	0.10	False
AllocateUninitializedArray - Duration of single invocation	155.52 ns	472.17 ns	3.04	0.11	False
AllocateArray - Duration of single invocation	178.93 ns	470.77 ns	2.63	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC<Byte>*'

Details

Payloads

Baseline
Compare

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7177633974690083 > 399.6578134529415.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -256.67053966083284 (T) = (0 -1722.6061554861167) / Math.Sqrt((56.16090401014884 / (15)) + (46.90461042453306 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.485184576382786 = (384.0658341145383 - 1722.6061554861167) / 384.0658341145383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.877226064390347 > 814.3475494617968.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -440.67129718721765 (T) = (0 -1874.8971003547222) / Math.Sqrt((11.847738386678492 / (15)) + (10.848146958465723 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.414850080560727 = (776.4031048748882 - 1874.8971003547222) / 776.4031048748882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 472.16513668101777 > 162.20111444090492.
IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -21.318217154468464 (T) = (0 -457.9533767902344) / Math.Sqrt((1.2117547655583252 / (15)) + (403.94823838655543 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.9559985600646497 = (154.92340996952944 - 457.9533767902344) / 154.92340996952944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.7704826208066 > 187.7223194932085.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -115.38793741220125 (T) = (0 -468.94846461482723) / Math.Sqrt((43.88253710633865 / (15)) + (6.639499228225897 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.5967833600469057 = (180.5882122590144 - 468.94846461482723) / 180.5882122590144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AllocateArray - Duration of single invocation	1.47 μs	2.28 μs	1.55	0.07	False
AllocateArray - Duration of single invocation	235.63 ns	797.53 ns	3.38	0.11	False
AllocateUninitializedArray - Duration of single invocation	186.48 ns	788.87 ns	4.23	0.09	False
AllocateUninitializedArray - Duration of single invocation	621.92 ns	2.38 μs	3.83	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC<Char>*'

Details

Payloads

Baseline
Compare

System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2832381637321504 > 1.5367268810257242.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -55.78707820712384 (T) = (0 -2298.0090499132884) / Math.Sqrt((63.27093197615269 / (15)) + (436.35815715226346 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.56747575912705 = (1466.0571536960056 - 2298.0090499132884) / 1466.0571536960056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 797.532396788991 > 250.09650954960767.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -164.6218631515267 (T) = (0 -800.9410409499897) / Math.Sqrt((1.3949408754027728 / (15)) + (23.237710032619965 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.371495773855683 = (237.56252259335443 - 800.9410409499897) / 237.56252259335443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 788.8731905186607 > 194.27387716710592.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -53.6701390096785 (T) = (0 -777.8408674296243) / Math.Sqrt((1.1605997238051653 / (15)) + (243.4243054817749 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.192164061129945 = (185.54638036278737 - 777.8408674296243) / 185.54638036278737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3790466059342608 > 660.7370015602506.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.929305140791712 (T) = (0 -2322.4811732723774) / Math.Sqrt((44.988564752216064 / (15)) + (6399.296344452154 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.6939609501352844 = (628.7238020714109 - 2322.4811732723774) / 628.7238020714109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReadAllBytesAsync - Duration of single invocation 📝 - Benchmark Source	274.91 μs	577.99 μs	2.10	0.27	False
ReadAllBytes - Duration of single invocation 📝 - Benchmark Source	202.83 μs	479.94 μs	2.37	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Details

Payloads

Baseline
Compare

System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 1048576)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 577.9861458333332 > 290.3141094388058.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -15.284701840479862 (T) = (0 -560749.06445802) / Math.Sqrt((487603873.49851245 / (15)) + (594233948.6783401 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.9797180198262446 = (283246.93660526245 - 560749.06445802) / 283246.93660526245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 479.94292338709687 > 231.82580033365883.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -23.577822242410264 (T) = (0 -490412.2564728353) / Math.Sqrt((491765563.06122065 / (15)) + (219213870.52027768 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.3457605174110605 = (209063.22398762486 - 490412.2564728353) / 209063.22398762486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 📝 - Benchmark Source	72.09 ns	84.47 ns	1.17	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Details

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.46788241404712 > 75.75313546496682.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -7.645247215071372 (T) = (0 -86.27122724084657) / Math.Sqrt((0.7910449094041205 / (15)) + (6.504105128688698 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.19203818135424594 = (72.37287243839447 - 86.27122724084657) / 72.37287243839447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_SourceGen_ - Duration of single invocation	139.24 μs	175.28 μs	1.26	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*'

Details

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.27994382022473 > 145.7066131121954.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -9.778227603968837 (T) = (0 -171882.92275646652) / Math.Sqrt((185977.0733591263 / (15)) + (23079504.215233866 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.2397050631655811 = (138648.23808782734 - 171882.92275646652) / 138648.23808782734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

Best guess for the commit range c4ff86d...4f933e1 is #87374.

cc @MichalPetryka

[AndyAyersMS]

This is also affecting x64:

It's possible that the intrinsifcation here broke static PGO ingestion, but that's just a guess.

[AndyAyersMS]

It's possible that the intrinsifcation here broke static PGO ingestion, but that's just a guess.

No, that doesn't seem to be the case. Main method in the benchmark above is TranscodeToUtf8 and

base

82.00% 4.265E+07 Tier-1 [System.Private.CoreLib]Utf8Utility.TranscodeToUtf8(wchar*,int32,unsigned int8*,int32,wchar*&,unsigned int8*&)
08.54% 4.44E+06 Tier-1 [System.Private.CoreLib]Utf16Utility.GetPointerToFirstInvalidChar(wchar*,int32,int64&,int32&)
07.25% 3.77E+06 native coreclr.dll

This method does not end up depending on GetArrayDataReference directly.

Another possibility here is the BDN version update we took with dotnet/performance#3120.

[AndyAyersMS]

Method	BDN	Toolchain	Input	Mean	Error	StdDev	Median	Min	Max	Ratio	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
GetBytes	Old BDN	\base-rel\corerun.exe	Cyrillic	123.3 us	1.10 us	0.97 us	123.3 us	121.9 us	125.0 us	1.00	31.9882	31.9882	31.9882	99.89 KB	1.00
GetBytes	Old BDN	\diff-rel\corerun.exe	Cyrillic	109.4 us	1.35 us	1.26 us	109.4 us	106.0 us	111.3 us	0.89	31.8966	31.8966	31.8966	99.89 KB	1.00
GetBytes	New BDN	\base-rel\corerun.exe	Cyrillic	155.06 us	2.152 us	2.013 us	155.07 us	152.11 us	158.99 us	1.00	0.00	32.1602	32.1602	32.1602	99.89 KB
GetBytes	New BDN	\diff-rel\corerun.exe	Cyrillic	154.99 us	1.623 us	1.519 us	155.06 us	152.45 us	157.82 us	1.00	0.02	32.1782	32.1782	32.1782	99.89 KB

Here

Old BDN is BenchmarkDotNet v0.13.2.2052-nightly
New BDN is BenchmarkDotNet v0.13.7-nightly.20230717.35

So that seems to be the most likely explanation for the "regression"

cc @cincuranet

[AndyAyersMS]

Also note the operation count dropped quite a bit. Note this can be an effect (not a cause) if the benchmark is indeed running more slowly, but it can also impact the timing.