[SPARK-31406][SQL][TEST] ThriftServerQueryTestSuite: Sharing test data and test tables among multiple test cases.

[beliefer]

What changes were proposed in this pull request?

This PR is related to #28060.
ThriftServerQueryTestSuite spend 17 minutes time to test.
I checked the code and found ThriftServerQueryTestSuite load test data repeatedly.
I've listed all the test cases order by time with desc in the hive-thriftserver module below.

Class	Spend time ↑	Failure	Skip	Pass	Total test case
ThriftServerQueryTestSuite	17 minutes	0	15	140	155
CliSuite	8 minutes 24 seconds	0	0	24	24
SparkThriftServerProtocolVersionsSuite	59 seconds	0	0	210	210
HiveThriftBinaryServerSuite	36 seconds	0	1	21	22
SparkMetadataOperationSuite	19 seconds	0	0	7	7
HiveCliSessionStateSuite	16 seconds	0	0	2	2
SparkSQLEnvSuite	16 seconds	0	0	1	1
HiveThriftHttpServerSuite	15 seconds	0	0	3	3
SingleSessionSuite	14 seconds	0	0	3	3
JdbcConnectionUriSuite	2.1 seconds	0	0	1	1
ThriftServerWithSparkContextSuite	1.4 seconds	0	0	1	1
SparkExecuteStatementOperationSuite	63 millseconds	0	0	2	2
UISeleniumSuite	-1 millseconds	0	1	0	1

I checked the code of ThriftServerQueryTestSuite and found ThriftServerQueryTestSuite load test data repeatedly.
This PR will improve the performance of ThriftServerQueryTestSuite.
Because #28060 provides createTestTables(

spark/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala

Line 574 in e42a394

privatedefcreateTestTables(session: SparkSession):Unit= {

) and removeTestTables(

spark/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala

Line 666 in e42a394

privatedefremoveTestTables(session: SparkSession):Unit= {

), this PR will still uses them.
The total time run ThriftServerQueryTestSuite before and after this PR show below.
Before

No	Time
1	18 minutes, 8 seconds
2	22 minutes, 44 seconds
3	17 minutes, 48 seconds
4	18 minutes, 30 seconds

After

No	Time
1	16 minutes, 11 seconds
2	17 minutes, 19 seconds
3	18 minutes, 15 seconds
4	17 minutes, 27 seconds

Why are the changes needed?

Improve the performance of ThriftServerQueryTestSuite.

Does this PR introduce any user-facing change?

'No'.

How was this patch tested?

Jenkins test

[SparkQA]

Test build #121068 has finished for PR 28180 at commit f2a3798.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[beliefer]

cc @cloud-fan@maropu

[cloud-fan]

thanks, merging to master/3.0!

[beliefer]

@cloud-fan Thank you.

[maropu]

late LGTM

[dongjoon-hyun]

Hi, All.
Unfortunately, the original SQLQueryTestSuite commit broke this ThriftServerQueryTestSuite in Maven environment in master and branch-3.0.

• https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-3.0-test-maven-hadoop-2.7-hive-2.3/211/

Please see the follow-up fix.

• [SPARK-31291][SQL][TEST][FOLLOWUP] Fix resource loading error in ThriftServerQueryTestSuite #28186

[juliuszsompolski]

@cloud-fan@beliefer could we do a followup for this suite to keep loading the data via JDBC? I think it was important part of the coverage.

[juliuszsompolski]

Thanks for fixing it to loading the data only once!
But loading the data via JDBC statement was part of the testing coverage, to test executing those updates via JDBC. Instead of removing loadTestData, could you override createTestTables with creating and loading the data via JDBC statements?
While doing that, could you use statement.executeUpdate instead of statement.execute?

[cloud-fan]

hmm, I think we already have SQL tests for DDL/DML commands? If not we should add them.

[juliuszsompolski]

There are some tests that run a "CREATE TABLE" from within the test to set up some local data, but I believe not an extensive coverage.

[cloud-fan]

shall we create a test file to include all the temp views here and test reading/dropping?

[cloud-fan]

or we want to test executeUpdate explicitly?