Skip to content

[BEAM-151] Add support for RunnableOnService validation tests to runners module#173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lukecwik wants to merge 3 commits into from
Closed

[BEAM-151] Add support for RunnableOnService validation tests to runners module #173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
60d5471
[BEAM-151] Add support for RunnableOnService validation tests to runn…
lukecwik Apr 13, 2016
86e61c5
[BEAM-151] !fixup Add the runnable-on-service-tests execution
lukecwik Apr 13, 2016
c1b4f33
[BEAM-151] !fixup Fix comment in TestPipeline
lukecwik Apr 13, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions examples/java/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -36,17 +36,6 @@

<packaging>jar</packaging>

<profiles>
<profile>
<id>DataflowPipelineTests</id>
<properties>
<runIntegrationTestOnService>true</runIntegrationTestOnService>
<testGroups>com.google.cloud.dataflow.sdk.testing.RunnableOnService</testGroups>
<testParallelValue>both</testParallelValue>
</properties>
</profile>
</profiles>

<build>
<plugins>
<plugin>
Expand Down
11 changes: 0 additions & 11 deletions examples/java8/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -37,17 +37,6 @@

<packaging>jar</packaging>

<profiles>
<profile>
<id>DataflowPipelineTests</id>
<properties>
<runIntegrationTestOnService>true</runIntegrationTestOnService>
<testGroups>com.google.cloud.dataflow.sdk.testing.RunnableOnService</testGroups>
<testParallelValue>both</testParallelValue>
</properties>
</profile>
</profiles>

<build>
<plugins>
<plugin>
Expand Down
7 changes: 0 additions & 7 deletions pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -289,13 +289,6 @@
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<parallel>${testParallelValue}</parallel>
<threadCount>4</threadCount>
<groups>${testGroups}</groups>
<systemPropertyVariables>
<runIntegrationTestOnService>${runIntegrationTestOnService}</runIntegrationTestOnService>
<projectName>${dataflowProjectName}</projectName>
</systemPropertyVariables>
<useManifestOnlyJar>false</useManifestOnlyJar>
<trimStackTrace>false</trimStackTrace>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
Expand Down
25 changes: 25 additions & 0 deletions runners/flink/examples/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,31 @@
<parallelism>1</parallelism>
</properties>

<profiles>
<profile>
<id>disable-runnable-on-service-tests</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>runnable-on-service-tests</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<dependencies>

<dependency>
Expand Down
24 changes: 24 additions & 0 deletions runners/flink/runner/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,30 @@

<packaging>jar</packaging>

<profiles>
<profile>
<id>disable-runnable-on-service-tests</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>runnable-on-service-tests</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<dependencies>
<!-- Flink dependencies -->
Expand Down
47 changes: 29 additions & 18 deletions runners/google-cloud-dataflow-java/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -21,14 +21,11 @@

<modelVersion>4.0.0</modelVersion>

<!-- We use the Beam parent to allow for runners/flink, examples/java and
sdks/java/maven-archetypes to depend on the Dataflow runner. This is
until those modules can migrate off of the Dataflow runner classes. -->
<parent>
<groupId>org.apache.beam</groupId>
<artifactId>parent</artifactId>
<version>0.1.0-incubating-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
<groupId>org.apache.beam</groupId>
<artifactId>runners-parent</artifactId>
<version>0.1.0-incubating-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>google-cloud-dataflow-java-runner</artifactId>
Expand All@@ -41,21 +38,35 @@
<beam.version>0.1.0-incubating-SNAPSHOT</beam.version>
<timestamp>${maven.build.timestamp}</timestamp>
<maven.build.timestamp.format>yyyy-MM-dd HH:mm</maven.build.timestamp.format>
<dataflow>com.google.cloud.dataflow</dataflow>
<runIntegrationTestOnService>false</runIntegrationTestOnService>
<testParallelValue>none</testParallelValue>
<testGroups></testGroups>
<dataflowProjectName></dataflowProjectName>
</properties>

<!-- Disable integration tests unless told configured by a test flag.

@davorbonacidavorbonaciApr 13, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This profile disables automatic execution of RunnableOnService integration tests on Google Cloud Dataflow service. However, the tests will be executed if runnableOnServicePipelineOptions is specified.

This allows users by default to opt-out of running Dataflow integration
tests because of the requirements to have a valid GCP project and so forth. -->
<profiles>
<profile>
<id>DataflowPipelineTests</id>
<properties>
<runIntegrationTestOnService>true</runIntegrationTestOnService>
<testGroups>com.google.cloud.dataflow.sdk.testing.RunnableOnService</testGroups>
<testParallelValue>both</testParallelValue>
</properties>
<id>disable-runnable-on-service-tests</id>
<activation>
<property>
<name>!runnableOnServicePipelineOptions</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>runnable-on-service-tests</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

Expand Down
31 changes: 31 additions & 0 deletions runners/pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -39,4 +39,35 @@
<module>spark</module>
</modules>

<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>

@davorbonacidavorbonaciApr 13, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version shouldn't be needed.

<executions>
<execution>
<id>runnable-on-service-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<groups>com.google.cloud.dataflow.sdk.testing.RunnableOnService</groups>
<parallel>all</parallel>

@davorbonacidavorbonaciApr 13, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

used to be both, not all. Just confirming it is intentional.

@lukecwiklukecwikApr 13, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, intentional. both has been deprecated

<threadCount>4</threadCount>
<dependenciesToScan>
<dependency>org.apache.beam:java-sdk-all</dependency>
</dependenciesToScan>
<systemPropertyVariables>
<beamTestPipelineOptions>${runnableOnServicePipelineOptions}</beamTestPipelineOptions>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
Loading