[SPARK-24916][SQL] Fix type coercion for IN expression with subquery

[wangyum]

What changes were proposed in this pull request?

The below SQL will throw AnalysisException. but it can success on Spark 2.1.x. This pr fix this issue.

CREATE TEMPORARY VIEW t4 ASSELECT*FROMVALUES
(CAST(1AS DOUBLE), CAST(2AS STRING), CAST(3AS STRING))
AS t1(t4a, t4b, t4c);
CREATE TEMPORARY VIEW t5 ASSELECT*FROMVALUES
(CAST(1ASDECIMAL(18, 0)), CAST(2AS STRING), CAST(3ASBIGINT))
AS t1(t5a, t5b, t5c);
SELECT*FROM t4
WHERE
(t4a, t4b, t4c) IN (SELECT t5a, t5b, t5c FROM t5);

I tested the different combinations of getting commonTypes, it looks like findCommonTypeForBinaryComparison + findWiderTypeWithoutStringPromotionForTwo is the best way.

No.	commonTypes	desc
1	findCommonTypeForBinaryComparison + findTightestCommonType	Can't compare double with decimal
2	findCommonTypeForBinaryComparison + findWiderTypeForTwo	Seems good
3	findWiderTypeForTwo	Can't compare string with double
4	findCommonTypeForBinaryComparison + findWiderTypeWithoutStringPromotionForTwo	Same to No.2
5	findWiderTypeWithoutStringPromotionForTwo	Can't compare string with int
6	findCommonTypeForBinaryComparison + findTightestCommonType + findWiderTypeForDecimal	Same to No.2

The details can be found in the commit log.

How was this patch tested?

unit tests

[mgaido91]

I think this is basically the same of what I proposed in #19635. Unfortunately, that PR got a bit stuck...

[HyukjinKwon]

Do we really need to test all the combinations? We need most of such logics should be tested in findWiderTypeWithoutStringPromotionForTwo and we could have just few end to end tests.

[wangyum]

Oh. It turns out that @dilipbiswal is talking about that PR. I didn't find it in your recent PR. Let’s wait if the test can pass.

[SparkQA]

Test build #93538 has finished for PR 21871 at commit 8ef142f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[wangyum]

workaround:

SELECT*FROM t4
WHERE
(t4a, t4b, t4c) IN (SELECT t5a, t5b, t5c FROM t5);
->SELECT*FROM t4
WHERE
(t4a, t4b, t4c) IN (SELECT CAST(t5a as DOUBLE), CAST(t5b AS STRING), CAST(t5c AS STRING) FROM t5);