Skip to content

Investigate benchmark regressions in Max, CompareTo, ReduceSignificance, Pow, and Divide since 2.0 #76

Description

@matt-edmondson

Summary

Converting PreciseNumber to a value type in #70 cut allocation on every operation that returns a value, and made most operations faster. A same-machine comparison against 1.x also showed a few operations getting slower. The full 88-benchmark table is in this comment on #72.

Benchmark Parameter 1.x 2.0 Change
Max 8 digits 2.991 ns 3.681 ns +23%
Max 30 digits 2.985 ns 3.486 ns +17%
Max 200 digits 2.301 ns 4.244 ns +84%
CompareTo 8 digits 1.645 ns 1.906 ns +16%
CompareTo 30 digits 1.650 ns 1.907 ns +16%
CompareTo 200 digits 1.834 ns 1.923 ns +5%
ReduceSignificance 8 digits 63.53 ns 75.44 ns +19%
Pow exponent 2 50.87 ns 57.33 ns +13%
TryFormat 8 digits 36.96 ns 40.90 ns +11%
Divide 8 digits 227.46 ns 237.26 ns +4%
Divide 30 digits 381.25 ns 405.16 ns +6%
Divide 200 digits 1,653.78 ns 1,684.76 ns +2%

Nothing is broken, and the absolute differences are small, but Max at 200 digits nearly doubled.

Caveats

Suggested next steps

  1. Rerun the comparison between 1.x (cd9a822) and current main, with both versions in one job. Separate workflow runs can land on different CPU generations, which made earlier numbers incomparable. Adding a baseline ref input to benchmarks.yml would make this repeatable. The one-off workflow used here checked out both refs side by side and passed --artifacts per class.
  2. Profile whichever regressions reproduce. Places to start:
    • Max is x > y ? x : y, which now returns a copy of a 24-byte struct where 1.x returned a reference.
    • CompareTo delegates to the static Compare, so check that it still inlines.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P4Backlog; nothing brokenenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions