[mono][jit] Fuse SIMD extract and insert on arm64 by jandupej · Pull Request #92714 · dotnet/runtime

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

jandupej · 2023-09-27T15:00:40Z

Arm64 has the ins instruction which is capable of inserting from a specific lane into another lane. This PR takes advantage of this and fuses constant extract with constant insert (or elementwise constructor). Only float and double vectors are affected, as the appropriate mono operations for int types are missing. This change affects only mini, not LLVM.

This code:

privatestaticVector128<float>DUT1(Vector128<float>x){returnVector128.Create(Vector128.GetElement(x,1),Vector128.GetElement(x,2),Vector128.GetElement(x,3),Vector128.GetElement(x,0));}

Would compile into:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c ldr q0, [x29, #0x20]
0000000000000010 dup.4s v4, v0[1]
0000000000000014 dup.4s v3, v0[2]
0000000000000018 dup.4s v2, v0[3]
000000000000001c dup.4s v1, v0[0]
0000000000000020 eor.16b v0, v0, v0
0000000000000024 mov.s v0[0], v4[0]
0000000000000028 mov.s v0[1], v3[0]
000000000000002c mov.s v0[2], v2[0]
0000000000000030 mov.s v0[3], v1[0]
0000000000000034 str q0, [x29, #0x10]
0000000000000038 ldp x0, x1, [x29, #0x10]
000000000000003c mov sp, x29
0000000000000040 ldp x29, x30, [sp], #0x30
0000000000000044 ret

Current codegen is:

0000000000000000 stp x29, x30, [sp, #-0x30]!
0000000000000004 mov x29, sp
0000000000000008 stp x0, x1, [x29, #0x20]
000000000000000c eor.16b v0, v0, v0
0000000000000010 ldr q1, [x29, #0x20]
0000000000000014 mov.s v0[0], v1[1]
0000000000000018 mov.s v0[1], v1[2]
000000000000001c mov.s v0[2], v1[3]
0000000000000020 mov.s v0[3], v1[0]
0000000000000024 str q0, [x29, #0x10]
0000000000000028 ldp x0, x1, [x29, #0x10]
000000000000002c mov sp, x29
0000000000000030 ldp x29, x30, [sp], #0x30
0000000000000034 ret

Addresses #85166.

SamMonoRT · 2023-09-27T20:16:19Z

/azp run

azure-pipelines · 2023-09-27T20:16:25Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

…n). Fixed bug.

fanyang-mono

LGTM!

jandupej · 2023-10-04T12:49:37Z

CI failures seem unrelated. Merging.

Fuse extract and insert into arm64 ins.
b409943

jandupej requested review from SamMonoRT, fanyang-mono, lambdageek and vargaz as code owners September 27, 2023 15:00

ghost assigned jandupej Sep 27, 2023

ghost added the area-Codegen-JIT-mono label Sep 27, 2023

jandupej added this to the 9.0.0 milestone Sep 27, 2023

fanyang-mono reviewed Sep 27, 2023
View reviewed changes

Comment threadsrc/mono/mono/mini/simd-intrinsics.c

Extending to vector4.
c8c340e

fanyang-mono reviewed Sep 27, 2023
View reviewed changes

Comment threadsrc/mono/mono/mini/simd-intrinsics.c Outdated

build-analysisBot mentioned this pull request Sep 27, 2023
[wasm] WBT - tests broken due to old skiasharp package reference #92743
Closed

jandupej added 4 commits September 29, 2023 11:47

Ins index is now a function. Removed newline.
0b73b3f

Refactoring.
5a0bb3b

SIMD extract ops have no side offects (to facilitate their eliminatio…
828103a
…n). Fixed bug.

Fixed arm32 build.
b388c46

jandupej linked an issue Oct 2, 2023 that may be closed by this pull request
[mono] Fuse Vector128.WithElement and Vector128.GetElement on arm64 #85166
Closed

jandupej requested a review from matouskozak October 2, 2023 13:14

steveisok requested a review from lateralusX October 2, 2023 15:04

fanyang-mono approved these changes Oct 2, 2023
View reviewed changes

jandupej merged commit 6486250 into dotnet:mainOct 4, 2023

jandupej deleted the arm64-fused-ins-ext branch October 4, 2023 12:49

build-analysisBot mentioned this pull request Oct 5, 2023
tracing/runtimeeventsource/nativeruntimeeventsource/nativeruntimeeventsource.sh Timed Out on Mono OSX x64 with interpreter #92733
Closed

ghost locked as resolved and limited conversation to collaborators Nov 3, 2023