[TOPI][CUDA] Fix Winograd Kernel Size Support

[comaniac]

The merged PR #4260 fixes the padding issue when building Winograd conv2d for CUDA, but we found the kernel size is still a problem.

The constraints of using Winograd on CUDA has been released in the previous PR #3553. Specifically, the original Winograd limits the kernel size (=3x3), padding (=1x1) and strides (=1x1). PR #3553 released them to square kernel size (e.g., 3x3, 5x5, 7x7), strides (=1x1), and arbitrary padding.

However, even PR #4260 fixes the miscalculation issue caused by padding size, the miscalculation caused by kernel size is still there. Here is a code snippet in conv2d_winograd.py after PR #4260:

ifnotpre_computed: # kernel tensor is raw tensor, do strict checkifdilation_h!=1ordilation_w!=1:
kernel=dilation(kernel, (1, 1, dilation_h, dilation_w))
CO, CI, KH, KW=get_const_tuple(kernel.shape)
assertHSTR==1andWSTR==1andKH==KWelse:
# kernel tensor is pre-transfomred. this op is created by alter op layout.# dilation is not supported_, _, CI, CO=get_const_tuple(kernel.shape)
KH=KW=3assertHSTR==1andWSTR==1anddilation_h==1anddilation_w==1

As can be seen, kernel size is forced to 3x3 in the pre-computed case, but it could be any size according to PR #3553. This PR supports the kernel size recovery as follows:

alpha, _, CI, CO=get_const_tuple(kernel.shape)
KH=KW=alpha+1-tile_size

Here is another pending issue that I haven't resolved in this PR: the fixed errors in #4260 as well as this PR should be detected by the unit tests but they weren't. The reaons is that the unit test doesn't cover the part with pre_computed=True, but I have no idea how to cover it.

@cbalint13@vinx13 could you review and suggest how to improve the unit test? Thanks.

[vinx13]

To test with pre_computed case, you can add a relay unit test that runs a single conv layer model under opt_level=3. You may need to use winograd fallback context

[comaniac]

Thanks for the suggestion. The unit test has been added and I've confirmed the unit test covers the desire part.

One miner issue is that I have to adjust the error tolerance to 1e-3. Here are the max absolute differences by different shapes:

Kernel 3x3, Padding 1: ~1.6e-5
Kernel 3x3, Padding 0: ~5e-5
Kernel 7x7, Padding 2: ~1.6e-4

I assume this was caused by floating point error, because the reference numpy implementation is not based on Winograd algorithm but a normal conv2d. Please help confirm.

[cbalint13]

Thanks for the suggestion. The unit test has been added and I've confirmed the unit test covers the desire part.

One miner issue is that I have to adjust the error tolerance to 1e-3. Here are the max absolute differences by different shapes:

Kernel 3x3, Padding 1: ~1.6e-5
Kernel 3x3, Padding 0: ~5e-5
Kernel 7x7, Padding 2: ~1.6e-4

I assume this was caused by floating point error, because the reference numpy implementation is not based on Winograd algorithm but a normal conv2d. Please help confirm.

@comaniac

• It is normal to have increased error when tile size are beyond (3x3) in case of winograd.

[comaniac]

@comaniac

• It is normal to have increased error when tile size are beyond (3x3) in case of winograd.

Thanks for the confirmation. Just want to make sure it's expected.