[BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.

[aaltay]

This is a follow up to https://github.com/apache/beam/pull/11744/files. Newly added pipeline option breaks these tests because contains_inanyorder requires all items to match the matcher list. Changed this to has_items which reverses the relationship and require each matcher to match an item.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Dataflow	Flink	Samza	Spark	Twister2
Go		---		---		---
Java
Python				---		---
XLang		---		---		---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[aaltay]

@TobKed - question related to failed "Build python wheels / Build wheels on macos-latest" (https://github.com/apache/beam/pull/12288/checks?check_run_id=879924787) I do not see any logs other that "This check failed" message. How do we debug this?

[TobKed]

"This check failed" is really weird, I never seen this before. Looks like GitHub Actions issue.
I found related issue: actions/runner-images#841

@aaltay do you have permission to rerun workflow? If not please push some changes e.g. rebase. It should re-trigger run and hopefully we will get more info.

[TobKed]

I found information:

The request: 1087213 was abandoned due to an infrastructure failure. Notification of assignment to a runner was never received.

It is visible in Annotations section here:
https://github.com/apache/beam/actions/runs/172037362

[aaltay]

@TobKed

• I was able to re-run the workflow, but they did not remove the failed runs for the github UI. It looked like a mix of failed/pending runs. I re-pushed to restart all tests.
• Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?
• Should we also file an issue with github? Could this be a quota issue?

[aaltay]

@tysonjh - Gradle is failing with "java.io.IOException: No space left on device" errors. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Commit/12004/

@chamikaramj@ihji test_expand_kafka_read and test_external_transform are failing on Flink. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Commit/5854/

[aaltay]

Restarting tests, since this failures are not related to the specific flake this PR is fixing. And triaged the issues in the previous comment.

[aaltay]

Run Portable_Python PreCommit

[aaltay]

Run Python2_PVR_Flink PreCommit

[pabloem]

Run Python2_PVR_Flink PreCommit

[aaltay]

@chamikaramj@ihji test_expand_kafka_read and test_external_transform are failing on Flink. Link to failing tests: (https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Phrase/304/) -- same error as before. This looks like an issue, I will find the JIRA issue or file a new one.

[aaltay]

Run Python2_PVR_Flink PreCommit

[aaltay]

@chamikaramj@ihji test_expand_kafka_read and test_external_transform are failing on Flink. Link to failing tests: (https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Phrase/304/) -- same error as before. This looks like an issue, I will find the JIRA issue or file a new one.

I think this is fixed by (#12303). (@ibzib to confirm.)

[aaltay]

Re-pushed to pick up (#12303)

[ibzib]

Now all the external transform tests in Python2_PVR_Flink are timing out instead. I'm thinking about either increasing the timeouts to these tests, or removing them since they might be redundant (XVR_Flink has its own postcommit already).

[aaltay]

@ibzib - May I ignore the failure and merge this change? I think this change is good (passing other tests). And it would be good to merge this as it address another flakiness.

[ibzib]

@ibzib - May I ignore the failure and merge this change? I think this change is good (passing other tests). And it would be good to merge this as it address another flakiness.

Yes, I think the failure is fine to ignore.

[TobKed]

@aaltay

• Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?

Sure. What is the process to do it?

• Should we also file an issue with github? Could this be a quota issue?

I see it is still happens: https://github.com/apache/beam/actions/runs/175737977
I think it is not related to the quota, at least not from project side. Seems like GitHub have some issues with infrastructure.
There is issue created: actions/runner-images#841 . I will observe it.

[aaltay]

@aaltay

• Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?

Sure. What is the process to do it?

Thank you.

There is really not much of a process. Anyone can signup and edit the wiki. After you signup/login, you can go to the page I linked, edit and add a small section about these tests. Does it make sense?

• Should we also file an issue with github? Could this be a quota issue?

I see it is still happens: https://github.com/apache/beam/actions/runs/175737977
I think it is not related to the quota, at least not from project side. Seems like GitHub have some issues with infrastructure.
There is issue created: actions/virtual-environments#841 . I will observe it.

[tysonjh]

@tysonjh - Gradle is failing with "java.io.IOException: No space left on device" errors. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Commit/12004/

This was fixed AFAICT by running some Jenkins jobs to clean up the local storage. Sorry about the delayed response.

[TobKed]

I created PR with documentation for GItHub Actions CI #12405