System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark has regressed

[adamsitnik]

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark has regressed, it might be related to #59415

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 net7.0 --filter System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark 

It seems that @performanceautofiler has missed the regression (cc @DrewScoggins ): https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark.html

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

Result	Base	Diff	Ratio	Modality	Operating System	Bit	Processor Name
Slower	4.67	8.10	0.58		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	3.72	7.00	0.53		Windows 11	X64	AMD Ryzen 9 5900X
Same	5.56	7.91	0.70		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	7.99	11.94	0.67	several?	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same	5.62	7.03	0.80		Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	5.04	6.85	0.74		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	5.20	8.06	0.65		Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	7.80	10.54	0.74		Windows 11	X64	Unknown processor
Same	9.87	9.32	1.06	multimodal	Windows 11	X64	Unknown processor
Slower	4.22	7.39	0.57		ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	5.40	10.62	0.51		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	9.77	12.18	0.80		centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	7.62	11.41	0.67		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	6.06	8.61	0.70		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	5.99	8.41	0.71		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	6.04	8.30	0.73		ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	10.94	11.56	0.95		Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Slower	7.16	11.15	0.64	bimodal	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	34.35	31.63	1.09		Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	7.07	11.71	0.60		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	6.35	10.57	0.60		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

cc @tannergooding

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

---

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark has regressed, it might be related to #59415

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 net7.0 --filter System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark 

It seems that @performanceautofiler has missed the regression (cc @DrewScoggins ): https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark.html

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

Result	Base	Diff	Ratio	Modality	Operating System	Bit	Processor Name
Slower	4.67	8.10	0.58		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	3.72	7.00	0.53		Windows 11	X64	AMD Ryzen 9 5900X
Same	5.56	7.91	0.70		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	7.99	11.94	0.67	several?	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same	5.62	7.03	0.80		Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	5.04	6.85	0.74		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	5.20	8.06	0.65		Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	7.80	10.54	0.74		Windows 11	X64	Unknown processor
Same	9.87	9.32	1.06	multimodal	Windows 11	X64	Unknown processor
Slower	4.22	7.39	0.57		ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	5.40	10.62	0.51		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	9.77	12.18	0.80		centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	7.62	11.41	0.67		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	6.06	8.61	0.70		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	5.99	8.41	0.71		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	6.04	8.30	0.73		ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	10.94	11.56	0.95		Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Slower	7.16	11.15	0.64	bimodal	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	34.35	31.63	1.09		Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	7.07	11.71	0.60		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	6.35	10.57	0.60		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

cc @tannergooding

Author:	adamsitnik
Assignees:	-
Labels:	area-System.Numerics, tenet-performance
Milestone:	-

[adamsitnik]

Same goes for System.Numerics.Tests.Perf_Matrix4x4.Transpose

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_ubuntu%2018.04%2fSystem.Numerics.Tests.Perf_Matrix4x4.Transpose.html

Details

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Slower	4.78	6.93	0.69	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Slower	3.73	6.99	0.53	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Slower	5.36	8.72	0.61	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Slower	8.56	11.67	0.73	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	5.76	7.27	0.79	+0		Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	5.31	6.99	0.76	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Slower	5.32	7.61	0.70	+0		Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Slower	7.89	10.23	0.77	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	7.19	9.15	0.78	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Slower	4.68	7.52	0.62	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Slower	5.86	10.62	0.55	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Slower	10.83	13.40	0.81	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	8.18	7.92	1.03	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Slower	5.90	8.69	0.68	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Slower	5.88	8.02	0.73	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Slower	5.93	9.20	0.64	+0	bimodal	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	12.77	12.40	1.03	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	6.47	10.07	0.64	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	30.33	33.24	0.91	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	7.54	11.13	0.68	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	6.42	12.87	0.50	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

[adamsitnik]

And System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark:

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark.html

[adamsitnik]

There are more, I won't list them all here.

[tannergooding]

I'll have to look at the codegen for these. They haven't been changed in a long time and so I expect this is something that's changed in CSE or similar.

Notably, they could/should likely be simplified to use the xplat helper intrinsics now, rather than individual AdvSimd/Sse code paths