[QNN] Conv2D operator by anijain2305 · Pull Request #3580 · apache/tvm

anijain2305 · 2019-07-18T20:54:57Z

Lowering of QNN Conv2D operation. We break the convolution into 4 terms as described in Option 1 here. Other relevant discussion is present at #2351

anijain2305 · 2019-07-18T20:56:12Z

cc @FrozenGene @tqchen @yzhliu

anijain2305 · 2019-08-02T18:55:18Z

+ }
+}
+
+/*


@FrozenGene @jackwish @u99127
While the Requantize and Legalize pass are going through final details, it will be useful to prefetch and look at QNN conv2d lowering. Please review and let me know your comments.

zhenhuaw-me

Thank you for ping. Suggesting to have if {NHWC} elif {NCHW} else {assert} when handling different memory layout. And, I am not sure dividing the compute into 4 terms is optimization-friendly, but that's out of this PR scope I think.

ps. Only looked into part of this PR, will recheck when the requantize op were merged. Feel free to ping please.

zhenhuaw-me · 2019-08-05T07:05:11Z

+ input_scale of the input quantized tensors. The zero point of the output
+ quantized tensor is 0. By default, the dtype of output is int32. Please also
+ refer to Requantize operator to understand how to scale back the int32
+ ouptut to (u)int8.


zhenhuaw-me · 2019-08-05T07:07:20Z

+ out_dtype="int32"):
+ r"""Quantized 2D convolution.
+
+ This operator convolves quantized weight with quantized data. The scale of


Can it be convolves quantized data with quantized weight? I know they are basically the same though...

Yup, that sounds better.

zhenhuaw-me · 2019-08-05T07:08:08Z

+ r"""Quantized 2D convolution.
+
+ This operator convolves quantized weight with quantized data. The scale of
+ the output quantized tensor is the product of the weight_scale and


Do we need to align the term weight and kernel to be one of them rather than mixed?

Done to kernel

zhenhuaw-me · 2019-08-05T07:12:57Z


 namespace tvm {
 namespace relay {
-


If we are moving code like this to headers, would it be better to have a dedicated PR which involves no extra functionality?

zhenhuaw-me · 2019-08-05T07:17:43Z

+
+ const auto in_shape = get_shape(0);
+ int batch_size, in_channels;
+ // NCHW layout


Do we need to check it? Maybe simply if NCHW else if NHWC else assert?

Personally, I'd like to discuss the layout handling a bit (won't block the PR). To me, I always prefer code like if condition 1: path 1; elif condition 2: path 2: else (condition 3): path 3 or assert, even if the input guarantees that condition 3 won't happen. I think the path 1 inside a if section is easier to read, and the last assert is to handle unexpected code typo. Anyway, that is out the scope of this PR :).

zhenhuaw-me · 2019-08-05T07:18:02Z

+
+ const auto kernel_shape = get_shape(1);
+ int out_channels, kernel_h, kernel_w;
+ // OIHW layout


similar to input layout handling.

zhenhuaw-me · 2019-08-05T07:20:50Z

+ Array<IndexExpr> pad_w({param->padding[1], param->padding[1]});
+
+ Array<Array<IndexExpr>> pad_width;
+ pad_width = {pad_n, pad_c, pad_h, pad_w};


Layout check and handling?

anijain2305 · 2019-08-09T00:48:17Z

@u99127 @FrozenGene @jackwish @tqchen This is ready for review.

anijain2305 · 2019-08-09T00:51:49Z

@tqchenqnn.conv2d shares infer type functionality with nn.conv2d. Therefore, I have transferred that piece of code from cc to a header file and converted it to template function. It can now take Conv2DAttrs, QnnConv2dAttrs. Let me know if that looks ok.

anijain2305 · 2019-08-12T02:20:52Z

Pinging again in case this was missed @u99127 @FrozenGene @jackwish @tqchen

zhenhuaw-me

CI reports that there is merge conflicts, so request changes still...

It is very glad that we are reaching this, the conditional optimization is really beneficial. Thank you for the impressive work @anijain2305 !

zhenhuaw-me · 2019-09-02T07:34:11Z

+ }
+ auto reduced_t3 = Sum(Cast(weight, Int(32)), axes_t3, false, false);
+
+ // Find the newshape depenging on NCHW/NHWC layout.


is it depending?

zhenhuaw-me · 2019-09-02T07:46:51Z

+
+ const auto in_shape = get_shape(0);
+ int batch_size, in_channels;
+ // NCHW layout


Personally, I'd like to discuss the layout handling a bit (won't block the PR). To me, I always prefer code like if condition 1: path 1; elif condition 2: path 2: else (condition 3): path 3 or assert, even if the input guarantees that condition 3 won't happen. I think the path 1 inside a if section is easier to read, and the last assert is to handle unexpected code typo. Anyway, that is out the scope of this PR :).

anijain2305 · 2019-09-03T05:28:03Z

@jackwish Thanks for the good words :) I incorporated your comments. Can you please review again?

zhenhuaw-me

LGTM. Glad to participate in this, thank you for the great work!

anijain2305 · 2019-09-03T16:11:28Z

@zhiics @vinx13 As jackwish has approved, can you please review?

zhiics

Overall LGTM, only left nit comment.

zhiics · 2019-09-03T21:05:50Z

 return CallNode::make(op, {fill_value}, Attrs(attrs), {});
 }

+static inline Expr Conv2D(Expr data, Expr weight, Array<IndexExpr> strides,


It looks this is the same as MakeConv2d, right?
If this is true, should we just keep one signature instead of having duplication. I am not strongly against this because it's obviously used by other cases as well.

Yeah, I kept it to follow other usecases. I guess this repetition might be because typically TVM does not want header and implementation linking problems. Will keep it Conv2D for now.

I don't have a strong feeling about this, actually I'm not sure why we prefer to copying this (and the others) here instead of adding declarations

I think linking should be fine. We can put TVM_DLL if needed. But anyway, we can keep it this way for now.

zhiics · 2019-09-03T21:06:46Z

+
+/*!
+ * Copyright (c) 2019 by Contributors
+ * \file nn.cc


wrong file name.

zhiics · 2019-09-03T21:08:08Z

+ */
+WorkloadType GetWorkload(const Array<tvm::relay::Type>& arg_types, const QnnConv2DAttrs* param) {
+ // Get conv parameters.
+ auto get_shape = [&](const Type& type) {


we don't actually need to capture anything here, right?

zhiics · 2019-09-03T21:12:10Z

+ // Since, this is integer division (floor), we can first multiply the data by the pool_size and
+ // then perform avg_pool2d. Reversing this causes inaccuracy due to floor division.
+ auto scaled_hw_t2 = Multiply(casted_t2, MakeConstantScalar(Int(32), kernel_h * kernel_w));
+ Array<IndexExpr> padding;


Can we just use?
Array<IndexExpr> padding({0, 0})

zhiics · 2019-09-03T21:14:58Z

+from tvm.relay.testing import create_workload
+from tvm.contrib import graph_runtime
+
+def run_infer_type(expr):


run_infer_type could be obtained from tvm.relay.testing as well.

vinx13 · 2019-09-04T00:29:50Z

 return CallNode::make(op, {fill_value}, Attrs(attrs), {});
 }

+static inline Expr Conv2D(Expr data, Expr weight, Array<IndexExpr> strides,


I don't have a strong feeling about this, actually I'm not sure why we prefer to copying this (and the others) here instead of adding declarations

Rebasing. Empty commit. Clang-format styling.

zhiics

LGTM

Rebasing. Empty commit. Clang-format styling.

anijain2305 changed the title ~~[QNN] Convolution 2D Implementation.~~[QNN] Conv2D operatorJul 18, 2019

anijain2305 mentioned this pull request Jul 18, 2019
[Relay] [Quantization] WIP - Protoyping the quantized convolution op #3367
Closed

anijain2305 force-pushed the qnn_conv2d branch from ed1195a to 6f83532Compare July 18, 2019 21:10

anijain2305 changed the title ~~[QNN] Conv2D operator~~[QNN] WIP - Conv2D operatorJul 18, 2019

anijain2305 force-pushed the qnn_conv2d branch 3 times, most recently from 4f6e6bf to 52a3c2bCompare July 23, 2019 23:29

anijain2305 force-pushed the qnn_conv2d branch from 52a3c2b to 63cf319Compare July 31, 2019 22:54

anijain2305 commented Aug 2, 2019
View reviewed changes

anijain2305 changed the title ~~[QNN] WIP - Conv2D operator~~[QNN] Conv2D operatorAug 2, 2019

anijain2305 force-pushed the qnn_conv2d branch 7 times, most recently from c35d314 to 020c123Compare August 7, 2019 22:26

zhenhuaw-me suggested changes Aug 8, 2019
View reviewed changes

anijain2305 force-pushed the qnn_conv2d branch 6 times, most recently from bc5b780 to 699e34cCompare August 9, 2019 00:46

anijain2305 force-pushed the qnn_conv2d branch from 699e34c to 03cc5a1Compare August 9, 2019 21:36

anijain2305 force-pushed the qnn_conv2d branch from 03cc5a1 to cadb1e5Compare August 16, 2019 21:34

anijain2305 force-pushed the qnn_conv2d branch from cadb1e5 to 8baa53eCompare August 30, 2019 20:52

zhenhuaw-me suggested changes Sep 2, 2019
View reviewed changes

anijain2305 force-pushed the qnn_conv2d branch 2 times, most recently from f5cedbd to 434f40dCompare September 3, 2019 05:25

zhenhuaw-me approved these changes Sep 3, 2019
View reviewed changes

Comment threadsrc/relay/qnn/util.h

anijain2305 force-pushed the qnn_conv2d branch from 434f40d to 7773c32Compare September 3, 2019 16:10

zhiics reviewed Sep 3, 2019
View reviewed changes

anijain2305 force-pushed the qnn_conv2d branch from 7773c32 to f2e7ec4Compare September 4, 2019 00:20

vinx13 approved these changes Sep 4, 2019
View reviewed changes

[QNN] Convolution 2D Implementation.
d3a54c2
Rebasing. Empty commit. Clang-format styling.

anijain2305 force-pushed the qnn_conv2d branch from f2e7ec4 to d3a54c2Compare September 4, 2019 01:36

zhiics approved these changes Sep 4, 2019
View reviewed changes

zhiics merged commit 0d4870c into apache:masterSep 4, 2019

zhiics added the status: accepted label Sep 4, 2019

This was referenced Sep 5, 2019
Do type checking for the input and kernel in the qnn conv2d #3904
Merged
Qnn fully connected #3910
Merged

MarisaKirisame pushed a commit to MarisaKirisame/tvm that referenced this pull request Sep 7, 2019
[QNN] Convolution 2D Implementation. (apache#3580)
9464962
Rebasing. Empty commit. Clang-format styling.

wweic pushed a commit to wweic/tvm that referenced this pull request Sep 16, 2019
[QNN] Convolution 2D Implementation. (apache#3580)
c3e5bef
Rebasing. Empty commit. Clang-format styling.

wweic pushed a commit to wweic/tvm that referenced this pull request Sep 16, 2019
[QNN] Convolution 2D Implementation. (apache#3580)
687f8ab
Rebasing. Empty commit. Clang-format styling.

wweic pushed a commit to neo-ai/tvm that referenced this pull request Sep 16, 2019
[QNN] Convolution 2D Implementation. (apache#3580)
9cf6590
Rebasing. Empty commit. Clang-format styling.

tqchen mentioned this pull request Nov 8, 2019
[RELEASE][DRAFT] TVM v0.6 Release candidate #4259
Closed

+ }
+}
+/*

Uh oh!

Conversation

anijain2305 commented Jul 18, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anijain2305 commented Jul 18, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhenhuaw-me left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anijain2305 commented Aug 9, 2019

Uh oh!

anijain2305 commented Aug 9, 2019

Uh oh!

anijain2305 commented Aug 12, 2019

Uh oh!

zhenhuaw-me left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anijain2305 commented Sep 3, 2019

Uh oh!

zhenhuaw-me left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anijain2305 commented Sep 3, 2019

Uh oh!

zhiics left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

anijain2305 commented Jul 18, 2019•
edited
Loading

anijain2305 commented Jul 18, 2019•
edited
Loading