[Relay][AutoTVM] Relay op strategy by icemelon · Pull Request #4644 · apache/tvm

icemelon · 2020-01-07T18:57:41Z

RFC & discussion: https://discuss.tvm.ai/t/relay-op-strategy/5247/18
Update AutoTVM log format: https://discuss.tvm.ai/t/autotvm-log-format/5630

Todo list

Add op strategy for conv2d & conv2d related ops (e.g., conv2d_nchwc, conv2d_winograd)
Add & update op strategy for conv2d_transpose, dense, bitpack, bitserial_conv2d, bitserial_dense
Clean up autotvm (after updating all old autotvm templates)
Rewrite TOPI test
Update AutoTVM related tutorials

Thanks @kevinthesun @comaniac for the help to this PR!

MarisaKirisame · 2020-01-23T23:38:51Z

When is conv2d gonna be here? Need it for training.

icemelon · 2020-01-23T23:39:43Z

x86 conv2d is already in the pr. working on cuda and more targets now

icemelon · 2020-01-29T20:28:31Z

I've added the strategy for all ops. We can start to review this PR since it's huge. Could you help review it?
@tqchen @kevinthesun @comaniac @masahi @MarisaKirisame @hlu1 @yzhliu @zhiics @ZihengJiang @merrymercy @vinx13 @FrozenGene @jroesch @jwfromm

comaniac

Leave some comments related to AutoTVM changes.

comaniac · 2020-01-29T23:47:38Z

 self._counter += 1
- self.update(target, workload, cfg)
+ self.update(target, wkl, cfg)
+ cfg.workload = wkl


Where is cfg.workload initialized? I didn't find a definition in ConfigSpace. Also what's the purpose to have a workload field in a config space?

This is only specific to ApplyGraphBest. The reason is complicated. Because ApplyGraphBest relies on the order of query, we cannot use relay.backend.compile_engine.select_implement to collect the autotvm workload as it may query more than once. Therefore, this is a temporary work around that we sneak in the workload in the return cfg. We can remove this part of logic after we make ApplyGraphBest no longer relies on the query order.
@kevinthesun

I see. Could we define self.workload in ConfigSpace and add your comment on it? So that we will remember to remove it in the future.

Laurawly · 2020-01-30T00:27:52Z


+ if temp.name == "pad_temp":
+ data = temp.op.input_tensors[0]
+ # TODO(@Laurawly): Do we need to schedule pad op here?


Yeah we need to schedule temp.

tqchen · 2020-01-31T23:30:33Z

cc @merrymercy @vinx13 @ZihengJiang @jwfromm please help review if you have time

…phub

icemelon · 2020-02-24T19:50:36Z

@merrymercy Now the tophub version number has been updated and VTA TSIM test time is also back to normal after the fix. Could you review the PR again?

* relay op strategy fix lint bitpack strategy bitserial_dense (neo-ai#6) * update strategy * address comments fix a few topi test Dense strategy (neo-ai#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (neo-ai#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (neo-ai#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (neo-ai#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (neo-ai#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (neo-ai#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <kevinthesunwy@gmail.com>

* relay op strategy fix lint bitpack strategy bitserial_dense (#6) * update strategy * address comments fix a few topi test Dense strategy (#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <kevinthesunwy@gmail.com>

* relay op strategy fix lint bitpack strategy bitserial_dense (#6) * update strategy * address comments fix a few topi test Dense strategy (#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (apache#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (apache#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <kevinthesunwy@gmail.com>

apivovarov · 2021-11-08T22:37:41Z

There are couple TODOs related to PR4644 - TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged

tvm/python/tvm/relay/op/nn/nn.py
Line 220 in d11bdcd
# TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged
tvm/python/tvm/relay/qnn/op/qnn.py
Line 348 in 4ffbdcd
# TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged

icemelon force-pushed the op-dispatch branch from 3d3984d to ff0ca78Compare January 8, 2020 22:41

icemelon mentioned this pull request Jan 9, 2020
[CONV3D] remove incorrect schedule for conv3d_ndhwc #4659
Closed

yzhliu mentioned this pull request Jan 11, 2020
[Arith] add SizeVar representing non-neg valued variable in a tensor shape #4684
Merged

icemelon force-pushed the op-dispatch branch 2 times, most recently from 6b4cb34 to d42c343Compare January 22, 2020 21:50

icemelon mentioned this pull request Jan 23, 2020
[TOPI] Remove cpp upsampling and resize op #4769
Merged

icemelon force-pushed the op-dispatch branch 2 times, most recently from a90ee23 to 02cd271Compare January 27, 2020 19:49

icemelon marked this pull request as ready for review January 29, 2020 20:20

icemelon requested review from Huyuwei and Laurawly as code owners January 29, 2020 20:20

icemelon changed the title ~~[Draft] Relay op strategy~~[WIP] Relay op strategyJan 29, 2020

comaniac reviewed Jan 29, 2020
View reviewed changes

Laurawly reviewed Jan 30, 2020
View reviewed changes

icemelon mentioned this pull request Jan 30, 2020
[Relay] Conv2D padding representation #4787
Merged

tqchen added the status: need review label Jan 31, 2020

icemelon force-pushed the op-dispatch branch from 8dd4a6b to 6c2d508Compare February 3, 2020 19:30

MarisaKirisame reviewed Feb 4, 2020
View reviewed changes

Comment threadpython/tvm/relay/backend/compile_engine.py

icemelon force-pushed the op-dispatch branch 2 times, most recently from 897eb64 to b233005Compare February 5, 2020 23:06

junrushao reviewed Feb 5, 2020
View reviewed changes

Comment threadinclude/tvm/relay/op_attr_types.h Outdated

icemelon force-pushed the op-dispatch branch from efcddbb to 6ad83feCompare February 6, 2020 00:04

jwfromm reviewed Feb 6, 2020
View reviewed changes

Comment threadpython/tvm/relay/op/strategy/hls.py Outdated
Comment threadpython/tvm/relay/op/strategy/x86.py Outdated

icemelon force-pushed the op-dispatch branch from 6ad83fe to d832dd6Compare February 6, 2020 20:13

zhiics reviewed Feb 7, 2020
View reviewed changes

ZihengJiang reviewed Feb 7, 2020
View reviewed changes

Comment threadpython/tvm/autotvm/task/space.py

icemelon added 13 commits February 23, 2020 16:55

fix cpptest
2f3c719

fix docs
0445e5a

change data structure name and api
e496ae0

address comments
ea3dfaa

lint
eb630ef

fix rebase err
6cf45b5

updates
8b0081a

fix winograd test
3510177

fix doc
cf43e16

rebase
f07d92a

upgrade tophub version number
59bd399

fix bug
5338a57

re-enable vta tsim test after tophub is upgraded
24ae797

icemelon force-pushed the op-dispatch branch from 630cbec to 24ae797Compare February 24, 2020 00:56

fix vta test to use the correct args so the config can be found in to…
4f1806a
…phub

yzhliu approved these changes Feb 24, 2020
View reviewed changes

merrymercy approved these changes Feb 24, 2020
View reviewed changes

merrymercy merged commit 623dd20 into apache:masterFeb 24, 2020

icemelon deleted the op-dispatch branch February 24, 2020 21:14

icemelon added status: accepted and removed status: need review labels Feb 25, 2020

w-zr mentioned this pull request Feb 29, 2020
Performance regression of quantization on CUDA after [Relay][AutoTVM] Relay op strategy (#4644) #4972
Closed

FrozenGene mentioned this pull request Mar 11, 2020
[QNN] Support 4D padding. #5036
Merged

ZihengJiang mentioned this pull request Sep 17, 2020
TVM v0.7 Release Note Candidate #6486
Closed

Uh oh!

Conversation

icemelon commented Jan 7, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarisaKirisame commented Jan 23, 2020

Uh oh!

icemelon commented Jan 23, 2020

Uh oh!

icemelon commented Jan 29, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

comaniacJan 29, 2020

Choose a reason for hiding this comment

Uh oh!

icemelonJan 30, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

comaniacJan 30, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LaurawlyJan 30, 2020

Choose a reason for hiding this comment

Uh oh!

tqchen commented Jan 31, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

icemelon commented Feb 24, 2020

Uh oh!

apivovarov commented Nov 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

icemelon commented Jan 7, 2020•
edited
Loading

icemelon commented Jan 29, 2020•
edited
Loading

icemelonJan 30, 2020•
edited
Loading