[BEAM-11665] Publish Beam GCP BOM by kileys · Pull Request #13737 · apache/beam

kileys · 2021-01-12T19:30:17Z

Create a Beam GCP BOM for projects that use Beam and Google Cloud Platform.
Create a Maven archetype using the new BOM and run examples on it
Add BOM validation to release jenkins test and validation script

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	Dataflow	Samza	Twister2
Go	---	---	---
Java
Python		---	---
XLang		---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---		---	---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

kileys · 2021-01-12T19:30:44Z

Run Gradle Publish

codecov · 2021-01-12T19:50:41Z

Codecov Report

Merging #13737 (c5b7295) into master (6675586) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@ Coverage Diff @@## master #13737 +/- ##
==========================================
- Coverage 82.75% 82.74% -0.01% 
==========================================
Files 466 466 Lines 57525 57525 ==========================================
- Hits 47604 47601 -3 - Misses 9921 9924 +3

Impacted Files	Coverage Δ
.../python/apache_beam/testing/test_stream_service.py	`88.63% <0.00%> (-4.55%)`	⬇️
sdks/python/apache_beam/utils/interactive_utils.py	`88.09% <0.00%> (-2.39%)`	⬇️
...che_beam/runners/interactive/interactive_runner.py	`90.90% <0.00%> (-1.82%)`	⬇️
sdks/python/apache_beam/internal/metrics/metric.py	`86.45% <0.00%> (-1.05%)`	⬇️
sdks/python/apache_beam/transforms/util.py	`95.66% <0.00%> (-0.18%)`	⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py	`89.69% <0.00%> (-0.16%)`	⬇️
...hon/apache_beam/runners/worker/bundle_processor.py	`93.70% <0.00%> (-0.13%)`	⬇️
sdks/python/apache_beam/dataframe/frames.py	`91.88% <0.00%> (+0.41%)`	⬆️
sdks/python/apache_beam/runners/direct/executor.py	`96.82% <0.00%> (+0.52%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6675586...c5b7295. Read the comment docs.

kileys · 2021-01-12T21:14:52Z

Run Gradle Publish

suztomo · 2021-01-13T17:35:58Z

@kileys Can I see the generated BOM somewhere?

kileys · 2021-01-20T02:15:37Z

@kileys Can I see the generated BOM somewhere?
I updated the artifact name, but you can see the pom here:

@suztomo
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-bom-gcp/2.28.0-SNAPSHOT/beam-sdks-java-bom-gcp-2.28.0-20210112.211655-2.pom

kileys · 2021-01-20T02:16:21Z

Run Gradle Publish

elharo

<url>http://beam.apache.org</url> should use https
2 <inceptionYear>2016</inceptionYear> should be 2021
<organizationUrl>http://www.apache.org</organizationUrl> should use https
<archive>http://www.mail-archive.com/dev%beam.apache.org</archive> should use https

kileys · 2021-01-20T17:57:37Z

Run Dataflow PostRelease

kileys · 2021-01-20T21:04:17Z

Run Dataflow PostRelease

kileys · 2021-01-20T22:33:33Z

Run Dataflow PostRelease

kileys · 2021-01-20T22:42:01Z

<url>http://beam.apache.org</url> should use https
2 <inceptionYear>2016</inceptionYear> should be 2021
<organizationUrl>http://www.apache.org</organizationUrl> should use https
<archive>http://www.mail-archive.com/dev%beam.apache.org</archive> should use https

@elharo
That's an outdated pom before I made the changes. Also, for #2, it's inception year not current year.

Here's the newest pom:
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-google-cloud-platform-bom/2.28.0-SNAPSHOT/beam-sdks-java-google-cloud-platform-bom-2.28.0-20210120.122741-1.pom

kileys · 2021-01-21T01:26:26Z

Run Dataflow PostRelease

kileys · 2021-01-21T01:26:39Z

R: @kennknowles

suztomo · 2021-01-22T22:14:23Z

I see it's working as expected. (Effective pom after I replaced the version iwth the latest release 2.27: https://gist.github.com/suztomo/ab8bf9dc2653ce2a886a5a57a0204eb2#file-gistfile1-txt-L190 )

I want Guava version to be more higher: #13740

udim · 2021-09-21T20:06:45Z

 -PgcsBucket=${USER_GCS_BUCKET:5} # skip 'gs://' prefix
+
+ echo "**************************************************************************"
+ echo "* Java mobile game on DataflowRunner using Beam GCP BOM: UserScore, HourlyTeamScore, Leaderboard"


Should this be included as part of the release validation spreadsheet?
In other words, is this required to pass to validate the RC?

kileys added 2 commits January 12, 2021 19:19

Publish beam gcp bom
90561ac

move beam bom to dep
8a7269c

refactor
b9d4d34

ignore bom in javadoc project
e49030d

elharo reviewed Jan 13, 2021
View reviewed changes

Comment threadsdks/java/bom/common.gradle Outdated
Comment threadsdks/java/bom/common.gradle Outdated
Comment threadsdks/java/bom/common.gradle Outdated

fix urls, rename artifact
8f8e296

suztomo mentioned this pull request Jan 14, 2021
[BEAM-11626] Guava version 25.1-jre for Hadoop/Cassandra and Guava version 30.1 (latest) for the rest #13740
Closed
4 tasks

kileys added 6 commits January 14, 2021 23:59

Merge branch 'master' of github.com:apache/beam into beam-bom
1768113

add new archetype project
f829c05

scripts to run examples with bom
a6c0e15

add task to run bom examples
c02ffe7

add maven archetype for beam gcp bom
969dfe7

add to release script
c5b7295

elharo reviewed Jan 20, 2021
View reviewed changes

kileys changed the title ~~[WIP] Publish Beam GCP BOM~~[BEAM-11665] Publish Beam GCP BOMJan 20, 2021

suztomo approved these changes Jan 22, 2021
View reviewed changes

suztomo mentioned this pull request Jan 25, 2021
[BEAM-11626] Upgrading Guava to 30.1-jre while keeping 25.1-jre for Hadoop/Cassandra modules #13804
Merged
4 tasks

kennknowles approved these changes Jan 25, 2021
View reviewed changes

kennknowles merged commit cec7f10 into apache:masterJan 25, 2021

suztomo mentioned this pull request Jan 28, 2021
[BEAM-11648] In preparation for new BigQuery sink, update GCP BOM #13765
Merged

udim reviewed Sep 21, 2021
View reviewed changes

Uh oh!

Conversation

kileys commented Jan 12, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

Uh oh!

kileys commented Jan 12, 2021

Uh oh!

codecovBot commented Jan 12, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

kileys commented Jan 12, 2021

Uh oh!

suztomo commented Jan 13, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kileys commented Jan 20, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

elharo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021

Uh oh!

kileys commented Jan 20, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kileys commented Jan 21, 2021

Uh oh!

kileys commented Jan 21, 2021

Uh oh!

suztomo commented Jan 22, 2021

Uh oh!

udimSep 21, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kileys commented Jan 12, 2021•
edited
Loading

codecovBot commented Jan 12, 2021•
edited
Loading

kileys commented Jan 20, 2021•
edited
Loading

elharo left a comment •
edited
Loading

kileys commented Jan 20, 2021•
edited
Loading