[BEAM-10670] Fix Kafka expansion issues introduced by PR14801

[boyuanzz]

This is a followup change regarding to #14801 (comment)

cc: @ibzib@iemejia

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

ValidatesRunner compliance status (on master branch)

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---			---		---
Java
Python	---			---		---
XLang				---		---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[boyuanzz]

Run Flink ValidatesRunner

[boyuanzz]

Run Samza ValidatesRunner

[boyuanzz]

Run Spark ValidatesRunner

[boyuanzz]

Run Spark StructuredStreaming ValidatesRunner

[boyuanzz]

Run Twister2 ValidatesRunner

[boyuanzz]

Run Python Flink ValidatesRunner

[boyuanzz]

Run Python Spark ValidatesRunner

[boyuanzz]

Run Java Flink PortableValidatesRunner Batch

[boyuanzz]

Run Java Flink PortableValidatesRunner Streaming

[boyuanzz]

Run Java Samza PortableValidatesRunner

[boyuanzz]

Run Java Spark v2 PortableValidatesRunner Streaming

[boyuanzz]

Run Java Spark v3 PortableValidatesRunner Streaming

[boyuanzz]

Run Java Spark PortableValidatesRunner Batch

[boyuanzz]

Run Java PreCommit

[boyuanzz]

Run Java Spark v2 PortableValidatesRunner Streaming

[boyuanzz]

Java Spark v2 PortableValidatesRunner Streaming Tests seems pretty flaky to me.

[boyuanzz]

Run Java Spark v2 PortableValidatesRunner Streaming

[boyuanzz]

Run Java Spark v2 PortableValidatesRunner Streaming

[boyuanzz]

Looking at the history, that's fairly flaky: https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark2_Streaming/

[boyuanzz]

Run Java Spark v2 PortableValidatesRunner Streaming

[ihji]

LGTM.

@ibzib Have any idea why Spark v2 PortableValidatesRunner Streaming test is so flaky?

[ibzib]

LGTM.

@ibzib Have any idea why Spark v2 PortableValidatesRunner Streaming test is so flaky?

Spark portable streaming was left in the proof-of-concept state, and I haven't had time to keep the tests green. At this point it's probably more trouble than it's worth. I'm going to disable the test until we have the resources to actively maintain it.

[ihji]

Run Java Spark v2 PortableValidatesRunner Streaming

[ihji]

Keep failing. Interestingly every time it fails the number of failed tests are different.

@ibzib Would this be safe to be merged with the failed Run Java Spark v2 PortableValidatesRunner Streaming test? Or should we retry until it's green?

[ibzib]

Keep failing. Interestingly every time it fails the number of failed tests are different.

@ibzib Would this be safe to be merged with the failed Run Java Spark v2 PortableValidatesRunner Streaming test? Or should we retry until it's green?

This test suite is too unreliable to provide a useful signal. It's fine to ignore it.

[boyuanzz]

I'll cherry-pick this into #14844 as well. Thanks for your help, Heejong and Kyle!