Automatically enable Runner v2 for pipelines that use cross-language transforms.

[chamikaramj]

…transforms

This fixes#21787

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Mention the appropriate issue in your description (for example: "addresses [BEAM-121] Add DisplayData for IO transforms #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment "fixes #" instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[chamikaramj]

R: @robertwb

[chamikaramj]

Run XVR_Dataflow PostCommit

[chamikaramj]

Dataflow CrossLanguageValidatesRunner PythonUsingJava Tests

[chamikaramj]

Dataflow CrossLanguageValidatesRunner JavaUsingPython Tests

[robertwb]

Do we have to add portable_job_submission as well, or is that not automatic for all Java pipelines?

[chamikaramj]

That's automatic for all Java UW pipelines:

beam/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java

Line 1055 in b8e2e85

experiments.add("use_portable_job_submission");

[robertwb]

Note that Jenkis is currently down, see https://issues.apache.org/jira/browse/INFRA-23358

[codecov]

Codecov Report

Merging #21788 (b8e2e85) into master (f131b2d) will increase coverage by 0.14%.
The diff coverage is 84.39%.

❗ Current head b8e2e85 differs from pull request most recent head efbce54. Consider uploading reports for the commit efbce54 to get more accurate results

@@ Coverage Diff @@## master #21788 +/- ##
==========================================
+ Coverage 74.00% 74.15% +0.14% 
==========================================
Files 698 698 Lines 92224 92411 +187 ==========================================
+ Hits 68253 68530 +277 + Misses 22720 22630 -90 
Partials 1251 1251 

Flag	Coverage Δ
go	50.93% <ø> (ø)
python	83.77% <84.39%> (+0.18%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...thon/apache_beam/ml/inference/pytorch_inference.py	0.00% <0.00%> (ø)
sdks/python/apache_beam/utils/counters.py	85.39% <66.66%> (-1.36%)	⬇️
...on/apache_beam/runners/direct/sdf_direct_runner.py	35.53% <75.00%> (ø)
...eam/transforms/py_dataflow_distribution_counter.py	91.42% <75.00%> (-4.87%)	⬇️
sdks/python/apache_beam/transforms/cy_combiners.py	62.90% <76.00%> (+4.50%)	⬆️
sdks/python/apache_beam/runners/common.py	88.68% <83.33%> (+0.74%)	⬆️
sdks/python/apache_beam/transforms/core.py	92.38% <90.47%> (+0.23%)	⬆️
sdks/python/apache_beam/typehints/batch.py	87.09% <94.11%> (+0.86%)	⬆️
sdks/python/apache_beam/ml/inference/base.py	95.06% <96.66%> (+1.36%)	⬆️
...thon/apache_beam/ml/inference/sklearn_inference.py	93.33% <97.05%> (+0.83%)	⬆️
... and 27 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f131b2d...efbce54. Read the comment docs.

[chamikaramj]

Retest this please

[chamikaramj]

Run Java PreCommit

[chamikaramj]

Dataflow CrossLanguageValidatesRunner PythonUsingJava Tests

[chamikaramj]

Dataflow CrossLanguageValidatesRunner JavaUsingPython Tests

[chamikaramj]

Run Java PreCommit

[chamikaramj]

Run Java PostCommit

[chamikaramj]

Run Java PreCommit

[chamikaramj]

(failures are unrelated)

[chamikaramj]

Run Java PreCommit

[chamikaramj]

Only failing test is "org.apache.beam.runners.flink.batch.ReshuffleTest.testEqualDistributionOnReshuffleAcrossMultipleStages" which is completed unrelated.

https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/5235/