Skip to content

[topi] add ARM v8.2 udot (uint8) support - #3978

Merged
yzhliu merged 8 commits into
apache:masterfrom
yzhliu:armint8
Oct 1, 2019
Merged

[topi] add ARM v8.2 udot (uint8) support#3978
yzhliu merged 8 commits into
apache:masterfrom
yzhliu:armint8

Conversation

@yzhliu

@yzhliuyzhliu commented Sep 20, 2019•
edited
Loading

Copy link
Copy Markdown
Member

Add uint8 intrinsic for ARM. Currently it is udot.v2i32.v8i8 which may have too small lanes. will add more later

@anijain2305@zhiics@vinx13@ZihengJiang

@yzhliuyzhliu changed the title Armint8[topi] add ARM v8.2 udot (uint8) supportSep 20, 2019
zhiics
zhiics reviewed Sep 23, 2019
View reviewed changes

@zhiicszhiics left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few minors, other LGTM

Comment threadtopi/python/topi/arm_cpu/tensor_intrin.py
Comment threadtopi/python/topi/generic/conv2d.py
Comment threadtopi/python/topi/generic/conv2d.py
@anijain2305

anijain2305 commented Sep 23, 2019

Copy link
Copy Markdown
Contributor

Before merging, it would be good if we can try 2 more optimizations

  • Currently, udot seems to be little slow (~1x speedup). Reasoning can be that we are not fully utilizing the fused accumulation. We should look at the assembly to double-check that.
  • Please try udot.v4i32.v16i8, that should quadruple the throughput compared to FP32.

@yzhliu

yzhliu commented Sep 28, 2019

Copy link
Copy Markdown
MemberAuthor

@anijain2305@zhiics please review again.

anijain2305
anijain2305 reviewed Sep 30, 2019
View reviewed changes
Comment threadtopi/python/topi/generic/conv2d.py Outdated
anijain2305
anijain2305 reviewed Sep 30, 2019
View reviewed changes
Comment threadtopi/python/topi/nn/conv2d.py
anijain2305
anijain2305 approved these changes Sep 30, 2019
View reviewed changes

@anijain2305anijain2305 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Minor comments.

@anijain2305

anijain2305 commented Sep 30, 2019

Copy link
Copy Markdown
Contributor

It will be good if we can share the performance speedup results.

@yzhliu

yzhliu commented Sep 30, 2019

Copy link
Copy Markdown
MemberAuthor

@tqchen could you check the ci instance? it shows "no space left"

@yzhliu

yzhliu commented Sep 30, 2019

Copy link
Copy Markdown
MemberAuthor

@anijain2305 The avg speedup is ~2.1x compared to fp32

@tqchen

tqchen commented Oct 1, 2019

Copy link
Copy Markdown
Member

ci issue fixed

@yzhliu

yzhliu commented Oct 1, 2019

Copy link
Copy Markdown
MemberAuthor

Thanks @anijain2305@zhiics@tqchen

@yzhliu
yzhliu merged commit 5cc1764 into apache:masterOct 1, 2019
anijain2305 pushed a commit to anijain2305/tvm that referenced this pull request Oct 17, 2019
@yzhliuAnimesh Jain
[topi] add ARM v8.2 udot (uint8) support (apache#3978)
bec18d5
* [topi] add ARM v8.2 udot (uint8) support
* fix test case
* fix common conv2d schedule
* add back fp32_time in test
* fix lint
* fix doc, add support for int32_lanes=4, signed int
* fix lint
* add ic_bn % 4 checker in schedule
wweic pushed a commit to neo-ai/tvm that referenced this pull request Oct 18, 2019
@yzhliuWei Chen
[topi] add ARM v8.2 udot (uint8) support (apache#3978)
37612b0
* [topi] add ARM v8.2 udot (uint8) support
* fix test case
* fix common conv2d schedule
* add back fp32_time in test
* fix lint
* fix doc, add support for int32_lanes=4, signed int
* fix lint
* add ic_bn % 4 checker in schedule
petrex pushed a commit to petrex/tvm that referenced this pull request Oct 29, 2019
Peter Yeh
Merge branch 'master' into benchmark
ed4c185
* master:
Fix split's last factor issue (apache#4044)
[COMMUNITY] ajtulloch -> committer (apache#4043)
[TOPI]Add op argwhere (apache#3994)
[topi] add ARM v8.2 udot (uint8) support (apache#3978)
[COMMUNITY] anijain2305 -> reviewer (apache#4036)
[QNN] Renaming dense operator. (apache#4033)
[Relay][Compile_engine] Int64 shape handling for outputs. (apache#4031)
Add dmlc-core to the list of installed header directories. (apache#4035)
[ARITH] migrate indexdiv/mod to floordiv/mod (apache#4008)
[Relay] Move prelude to text format (apache#3939)
make tvm compilable by gcc 4.9.2 (apache#4032)
[AUTOTVM][DOCS] Add a link to the defining network description of auto-tuning tutorial (apache#4023)
[ARITH] cleanup the indexmod/div on python side (apache#4028)
[Fix] Add more pad_mode support for onnx converter (apache#4029)
Add parser support for ReLU tflite operator (apache#4022)
Additional MXNet Convolution and Deconvolution tests (apache#4026)
docs: minor spelling tweaks (apache#4027)
@tqchentqchen mentioned this pull request Nov 8, 2019
Closed
@masahimasahi mentioned this pull request Mar 24, 2022
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhiicszhiicszhiics approved these changes
@anijain2305anijain2305anijain2305 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yzhliu@anijain2305@tqchen@zhiics