Skip to content

[BEAM-254] Add apache-rat-plugin execution and fix files with missing…#297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
asfgit merged 1 commit into from
May 31, 2016
Merged

[BEAM-254] Add apache-rat-plugin execution and fix files with missing… #297

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
<!--

@swegnerswegnerMay 6, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these comments show up in the github pull request description field?

@jbonofrejbonofreMay 6, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's hidden in github (markdown).

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

Expand Down
17 changes: 17 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,20 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

language: java

sudo: false
Expand Down
19 changes: 19 additions & 0 deletions README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Apache Beam

[Apache Beam](http://beam.incubator.apache.org) is a unified model for defining both batch and streaming data-parallel processing pipelines, as well as a set of language-specific SDKs for constructing pipelines and Runners for executing them on distributed processing backends like [Apache Spark](http://spark.apache.org/), [Apache Flink](http://flink.apache.org), and [Google Cloud Dataflow](http://cloud.google.com/dataflow).
Expand Down
19 changes: 19 additions & 0 deletions examples/java/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Example Pipelines

The examples included in this module serve to demonstrate the basic
Expand Down
18 changes: 18 additions & 0 deletions examples/java/src/main/java/org/apache/beam/examples/complete/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# "Complete" Examples

Expand Down
18 changes: 18 additions & 0 deletions examples/java/src/main/java/org/apache/beam/examples/cookbook/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# "Cookbook" Examples

Expand Down
18 changes: 18 additions & 0 deletions examples/java8/src/main/java/org/apache/beam/examples/complete/game/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,21 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# 'Gaming' examples

Expand Down
42 changes: 42 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -219,8 +219,49 @@
</execution>
</executions>
</plugin>

<plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
<version>0.11</version>
<executions>
<execution>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
<configuration>
<reportFile>${project.build.directory}/${project.build.finalName}.rat</reportFile>
<excludeSubProjects>false</excludeSubProjects>
<useDefaultExcludes>true</useDefaultExcludes>
<excludes>
<!-- Keep exclude sync with .gitignore -->
<exclude>**/target/**/*</exclude>
<exclude>**/dependency-reduced-pom.xml</exclude>
<exclude>**/*.iml</exclude>
<exclude>**/package-list</exclude>
<exclude>**/user.avsc</exclude>
<exclude>**/test/resources/**/*.txt</exclude>
<exclude>**/test/**/.placeholder</exclude>
<exclude>.repository/**/*</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</pluginManagement>

<plugins>

@davorbonacidavorbonaciMay 23, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty line between pluginManagement and plugins sections?

@jbonofrejbonofreMay 23, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

<plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
</plugins>
</build>
</profile>

Expand All@@ -233,6 +274,7 @@
<dataflow.javadoc_opts>-Xdoclint:-missing</dataflow.javadoc_opts>
</properties>
</profile>

</profiles>

<dependencyManagement>
Expand Down
19 changes: 19 additions & 0 deletions runners/flink/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

Flink Beam Runner (Flink-Runner)
-------------------------------

Expand Down
19 changes: 19 additions & 0 deletions runners/spark/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

Spark Beam Runner (Spark-Runner)
================================

Expand Down
15 changes: 0 additions & 15 deletions sdks/java/build-tools/src/main/resources/beam/checkstyle.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,21 +25,6 @@ page at http://checkstyle.sourceforge.net/config.html -->

<module name="Checker">

<!-- Requires the Apache copyright header in each file.
Code intended to be open-sourced may have a multi-line copyright
notice, so that this required text appears on the second line:
<pre>
/*
* Licensed to the Apache Software Foundation (ASF) under one
* ...
</pre>
See http://checkstyle.sf.net/config_header.html -->
<module name="Header">
<property name="headerFile" value="${checkstyle.header.file}"/>
<property name="ignoreLines" value="2"/>
<property name="fileExtensions" value="java, scala"/>
</module>

<module name="FileTabCharacter">
<!-- Checks that there are no tab characters in the file. -->
</module>
Expand Down
17 changes: 0 additions & 17 deletions sdks/java/build-tools/src/main/resources/beam/header-file.txt
View file
Open in desktop

This file was deleted.

23 changes: 22 additions & 1 deletion sdks/java/core/src/main/proto/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,29 @@
## Protocol Buffers in Google Cloud Dataflow
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

This directory contains the Protocol Buffer messages used in Google Cloud
Dataflow.

## Protocol Buffers in Apache Beam

@swegnerswegnerMay 6, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be Google Dataflow specific, based on the below maven artifact reference.

@jbonofrejbonofreMay 6, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, does it mean that it should be removed from the Apache repo ?

@swegnerswegnerMay 6, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the only thing here (proto2_coder_test_messages.proto) is for testing. If that's the case, I would suggest updating the verbage of this README with the test details and removing dataflow-specific bits. At any rate, it might be easier to save for a separate PR as to not hold up this one.

Adding a few people that might know more: @dhalperi@kennknowles

@dhalperidhalperiMay 20, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a few options forward here, but yes I think that this can wait for another PR.

Right now we have not yet re-deployed the proto jar under org.apache.beam

@dhalperidhalperiMay 20, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbonofre : this is needed for testing ProtoCoder, which is currently part of the SDK core


This directory contains the Protocol Buffer messages used in Apache Beam.

They aren't, however, used during the Maven build process, and are included here
for completeness only. Instead, the following artifact on Maven Central contains
the binary version of the generated code from these Protocol Buffers:
Expand Down
19 changes: 18 additions & 1 deletion sdks/java/core/src/main/resources/org/apache/beam/sdk/sdk.properties
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,21 @@
# SDK source version.
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SDK source version

version=${pom.version}

build.date=${timestamp}
Expand Down
Loading