Skip to content

[Perf] Regressions in BenchmarksGame.FannkuchRedux_5 #68822

Description

@performanceautofiler

Run Information

Architecturex64
OSubuntu 18.04
Baseline8006e6a89bc02e410331e6323e3f6321b224b327
Comparee4163ea55ebb3673c29e1c2a850a6a790029d278
DiffDiff

Regressions in BenchmarksGame.FannkuchRedux_5

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
RunBench - Duration of single invocation23.95 ms26.24 ms1.100.01True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.FannkuchRedux_5*'
Details

Payloads

Baseline
Compare

Histogram

BenchmarksGame.FannkuchRedux_5.RunBench(n: 10, expectedSum: 38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.23831273571429 > 25.0471650415.
IsChangePoint: Marked as a change because one of 4/26/2022 12:44:27 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsRegressionStdDev: Marked as regression because -26.310636692824072 (T) = (0 -25737862.459520478) / Math.Sqrt((86587725068.70877 / (27)) + (81681730737.5921 / (36))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (36) - 2, .025) and -0.08183666563172187 = (23790894.94484017 - 25737862.459520478) / 23790894.94484017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions