Uh oh!
There was an error while loading. Please reload this page.
[BEAM-14334] Remove remaining forkEvery 1 from all Spark tests and stop mixing unit tests with runner validations. - #17662
Conversation
…op mixing unit tests with runner validations.
mosche
commented
May 13, 2022
R: @aromanenko-dev |
| project(":sdks:java:core").sourceSets.test.output.classesDirs, | ||
| project(":runners:core-java").sourceSets.test.output.classesDirs, | ||
| ) | ||
| testClassesDirs += files(project.sourceSets.test.output.classesDirs) |
There was a problem hiding this comment.
All unit tests should be run as part of the test task
| maxParallelForks 4 | ||
| useJUnit { | ||
| includeCategories 'org.apache.beam.sdk.testing.ValidatesRunner' | ||
| includeCategories 'org.apache.beam.runners.spark.UsesCheckpointRecovery' |
There was a problem hiding this comment.
as said above, tests of this custom category are normal unit tests and are already run during test. there's no runner validation for such a category
aromanenko-dev
commented
May 16, 2022
Run Spark ValidatesRunner |
aromanenko-dev
commented
May 16, 2022
Run Spark StructuredStreaming ValidatesRunner |
aromanenko-dev
left a comment
There was a problem hiding this comment.
Thanks! It LGTM, just several minor questions, ptal
| examplesJavaIntegrationTest | ||
| } | ||
| def sparkTestProperties(overrides = [:]) { |
There was a problem hiding this comment.
Is it just refactoring (extract method)?
| } | ||
| maxParallelForks 4 | ||
| useJUnit { |
There was a problem hiding this comment.
previously these unit tests were excluded during test runs because they failed. instead these were run as part of validatesRunnerStreaming.
I fixed them so they can be run as normal unit tests, so this exclusion isn't needed any more.
| maxParallelForks 4 | ||
| useJUnit { | ||
| includeCategories 'org.apache.beam.sdk.testing.ValidatesRunner' | ||
| includeCategories 'org.apache.beam.runners.spark.UsesCheckpointRecovery' |
mosche
commented
May 17, 2022
Run Spark ValidatesRunner |
1 similar comment
mosche
commented
May 17, 2022
Run Spark ValidatesRunner |
mosche
commented
May 17, 2022
Run Java PreCommit |
1 similar comment
mosche
commented
May 17, 2022
Run Java PreCommit |
mosche
commented
May 17, 2022
precommit issues are unrelated |
aromanenko-dev
commented
May 17, 2022
Agree, it's not related to this PR |
This is a follow up of #17406 fixing the same issue for all remaining testing tasks. That is basically the hadoop compatibility tests and the runner validations.
Additionally this removes the current intransparent mixing of unit tests and runner validations. All tests of the project are run as part of the
testtask (and the hadoop compatibility tests), whereas runner validations are defined intestingonly.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username).[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.