[SPARK-31732][TESTS] Disable some flaky tests temporarily

[cloud-fan]

What changes were proposed in this pull request?

It's quite annoying to be blocked by flaky tests in several PRs. This PR disables them. The tests come from 3 PRs I'm recently watching:
#28526
#28463
#28517

Why are the changes needed?

To make PR builder more stable

Does this PR introduce any user-facing change?

no

How was this patch tested?

N/A

[cloud-fan]

cc @dongjoon-hyun@HyukjinKwon@gatorsmile

[dongjoon-hyun]

Hi, @cloud-fan . So, you are targeting this PR only in master/3.0. Did I understand correctly?

BTW, the most annoying failures came from amp-jenkins-worker-05's corrupted local Maven repository. Hopefully, we can clean up that machine today.

[dongjoon-hyun]

Do you know how frequently it fail? In master Jenkins build, the following failures occur, too.

• org.apache.spark.scheduler.BarrierTaskContextSuite.global sync by barrier() call
• org.apache.spark.sql.kafka010.KafkaMicroBatchV1SourceSuite.subscribing topic by pattern with topic deletions

[SparkQA]

Test build #122684 has finished for PR 28547 at commit cd12026.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

In another PR (#28509 (comment)), BarrierTaskContextSuite.support multiple barrier() call within a single task.

org.apache.spark.scheduler.BarrierTaskContextSuite.support multiple barrier() call within a single task
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: 1031 was not less than or equal to 1000
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)

• https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122655/testReport/org.apache.spark.scheduler/BarrierTaskContextSuite/support_multiple_barrier___call_within_a_single_task/

[HyukjinKwon]

@dongjoon-hyun, actually some of them are not related to the corrupt local repository. This was in amp-jenkins-worker-03
https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/5029/.
Looks the corrupt repo was cleaned at amp-jenkins-worker-05 assuming the dependeny tests pass at https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/5039/ (?).

I am retriggering the builds for now. Let's disable them and run the RC if they are still frequently failed.

[cloud-fan]

@dongjoon-hyun I'll disable these tests as well, please let me know if you see more.

[dongjoon-hyun]

Thanks. @cloud-fan . I'm +1 for disabling the above, too.

Yes, @HyukjinKwon . The above are really flaky tests. What I meant was amp-jenkins-worker-05 failure was independent annoying stuff since there is no way to investigate more.

[HyukjinKwon]

retest this please

[HyukjinKwon]

LGTM

[SparkQA]

Test build #122710 has finished for PR 28547 at commit cd12026.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #122718 has finished for PR 28547 at commit 01aeeba.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

retest this please

[SparkQA]

Test build #122722 has finished for PR 28547 at commit 01aeeba.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

+1, LGTM. Thank you all.
Merged to master/3.0.

[dongjoon-hyun]

cc @rxin since he is a release manager of 3.0.0.
(Also, cc @gatorsmile )