Skip to content

[Perf] Linux/arm64: 8 Regressions on 10/25/2022 10:16:52 PM #77546

Description

@performanceautofiler

Run Information

Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.Tests.Perf_Int32

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
Parse - Duration of single invocation23.29 ns24.63 ns1.060.02False
TryParseSpan - Duration of single invocation21.90 ns23.15 ns1.060.04False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int32.Parse(value: "-2147483648")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.632511600690595 > 24.417097672605156.
IsChangePoint: Marked as a change because one of 8/17/2022 2:19:52 AM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -21.367659830262333 (T) = (0 -24.78464680275257) / Math.Sqrt((0.004289588449461454 / (33)) + (0.025921496883138544 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.06706362768607209 = (23.226962441310164 - 24.78464680275257) / 23.226962441310164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")
```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 23.148379235826216 > 22.916317556003428.
IsChangePoint: Marked as a change because one of 9/21/2022 2:58:31 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -16.118828279001892 (T) = (0 -23.265440551407238) / Math.Sqrt((0.03313309523689312 / (33)) + (0.038461681043579714 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.06907133709889778 = (21.76229007742539 - 23.265440551407238) / 21.76229007742539 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
Count - Duration of single invocation2.04 ms2.19 ms1.070.00False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.1899980488095236 > 2.1356328921666665.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -248.87284059613955 (T) = (0 -2188936.53581746) / Math.Sqrt((8463003.27484223 / (33)) + (632360.2174678757 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.07568077284491133 = (2034931.357960653 - 2188936.53581746) / 2034931.357960653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.MathBenchmarks.Double

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
Max - Duration of single invocation3.36 μs4.70 μs1.400.01False
Min - Duration of single invocation3.36 μs4.71 μs1.400.01False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.Double.Max

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.704199704803537 > 3.5229303271877956.
IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 10/7/2022 4:12:34 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -719.3497339728096 (T) = (0 -4702.373244418586) / Math.Sqrt((1.1486803895818931 / (33)) + (17.35879314825899 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.4014632318866617 = (3355.3311549160026 - 4702.373244418586) / 3355.3311549160026 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Double.Min
```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.706557142677909 > 3.5229940892914695.
IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 10/7/2022 4:12:34 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -380.9572996328128 (T) = (0 -4702.195851311515) / Math.Sqrt((0.835597943057459 / (33)) + (62.38099248108442 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.40145406150501944 = (3355.226532549939 - 4702.195851311515) / 3355.226532549939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.MathBenchmarks.Single

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
Max - Duration of single invocation3.35 μs4.62 μs1.380.01False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.Single.Max

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.617663316824365 > 3.515212398618343.
IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 10/7/2022 4:12:34 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -65.18343381765234 (T) = (0 -4637.62723895371) / Math.Sqrt((4.1579468182330235 / (33)) + (1955.8646465061574 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3851040978159002 = (3348.21566571534 - 4637.62723895371) / 3348.21566571534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.Text.Json.Tests.Perf_Strings

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
WriteStringsUtf16 - Duration of single invocation14.92 ms15.88 ms1.060.00False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 15.880320466666667 > 15.599360721102942.
IsChangePoint: Marked as a change because one of 8/6/2022 1:07:13 AM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -37.992647858158016 (T) = (0 -15801376.107222224) / Math.Sqrt((2421667831.8231554 / (33)) + (2666971929.5875583 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.06295566392693529 = (14865508.170723071 - 15801376.107222224) / 14865508.170723071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecturearm64
OSubuntu 20.04
Baselineafdd009ff95ea3da2e5b34f09f4a00c548f9be33
Comparea8ea8e67dc20689aa2e4c738a2f84ea3483ec872
DiffDiff

Regressions in System.Text.Perf_Utf8Encoding

BenchmarkBaselineTestTest/BaseTest QualityEdge DetectorBaseline IRCompare IRIR RatioBaseline ETLCompare ETL
GetByteCount - Duration of single invocation9.57 μs10.34 μs1.080.02False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.341321338300219 > 10.04419053792866.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -780.9666262078102 (T) = (0 -10342.405701166532) / Math.Sqrt((5.827832140118616 / (33)) + (4.042712192992345 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.08102042839598153 = (9567.262032700526 - 10342.405701166532) / 9567.262032700526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions