Skip to content

[BEAM-13430] Remove propdeps and replace with compileOnly - #16308

Merged
TheNeuralBit merged 10 commits into
apache:masterfrom
dpcollins-google:gradle7
Dec 21, 2021
Merged

[BEAM-13430] Remove propdeps and replace with compileOnly#16308
TheNeuralBit merged 10 commits into
apache:masterfrom
dpcollins-google:gradle7

Conversation

@dpcollins-google

@dpcollins-googledpcollins-google commented Dec 21, 2021

Copy link
Copy Markdown
Contributor
  • Still generate provided in the output POM.
  • Remove some unnecessary "provided" dependencies from examples.
  • add a dependency on runners:core-java from runners:java-fn-execution which was previously implicit
  • Remove usage of JacksonFactory which is deprecated, replace with GsonFactory
  • Fix dependencies of sdks:java:extensions:sql and other packages with issues revealed by this change

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

ValidatesRunner compliance status (on master branch)

LangULRDataflowFlinkSamzaSparkTwister2
Go---Build StatusBuild StatusBuild StatusBuild Status---
JavaBuild StatusBuild Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Python---Build Status
Build Status
Build Status
Build Status
Build Status
Build StatusBuild Status---
XLangBuild StatusBuild StatusBuild StatusBuild StatusBuild Status---

Examples testing status on various runners

LangULRDataflowFlinkSamzaSparkTwister2
Go---------------------
Java---Build Status
Build Status
Build Status
---------------
Python---------------------
XLang---------------------

Post-Commit SDK/Transform Integration Tests Status (on master branch)

GoJavaPython
Build StatusBuild StatusBuild Status
Build Status
Build Status

Pre-Commit Tests Status (on master branch)

---JavaPythonGoWebsiteWhitespaceTypescript
Non-portableBuild Status
Build Status
Build Status
Build Status
Build Status
Build StatusBuild StatusBuild StatusBuild Status
Portable---Build StatusBuild Status---------

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@codecov

codecovBot commented Dec 21, 2021

Copy link
Copy Markdown

Codecov Report

Merging #16308 (7afea6c) into master (c4dd8f4) will increase coverage by 37.52%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## master #16308 +/- ##
===========================================
+ Coverage 46.15% 83.68% +37.52% 
===========================================
Files 197 447 +250 Lines 19522 61715 +42193 ===========================================
+ Hits 9011 51645 +42634 - Misses 9540 10070 +530 + Partials 971 0 -971 
Impacted FilesCoverage Δ
sdks/go/pkg/beam/util/shimx/generate.go
sdks/go/pkg/beam/core/graph/coder/registry.go
sdks/go/pkg/beam/core/runtime/harness/statemgr.go
sdks/go/pkg/beam/core/metrics/store.go
sdks/go/pkg/beam/transforms/stats/count.go
sdks/go/pkg/beam/io/textio/textio.go
sdks/go/pkg/beam/io/filesystem/memfs/memory.go
sdks/go/pkg/beam/internal/errors/errors.go
sdks/go/pkg/beam/core/funcx/fn.go
sdks/go/pkg/beam/core/runtime/exec/status.go
... and 634 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4dd8f4...7afea6c. Read the comment docs.

Comment threadsdks/java/extensions/sql/datacatalog/build.gradle Outdated
@TheNeuralBit

Copy link
Copy Markdown
Member

retest this please

1 similar comment
@TheNeuralBit

Copy link
Copy Markdown
Member

retest this please

Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
@dpcollins-google

Copy link
Copy Markdown
ContributorAuthor

retest this please

1 similar comment
@TheNeuralBit

Copy link
Copy Markdown
Member

retest this please

Still generate <scope>provided</scope> in the output POM
Still generate <scope>provided</scope> in the output POM
@TheNeuralBit

Copy link
Copy Markdown
Member

Change to SQL test for BEAM-13506 LGTM

@TheNeuralBitTheNeuralBit left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Noting here that @dpcollins-google locally verified that generated poms still include provided scope where necessary.

@TheNeuralBit

Copy link
Copy Markdown
Member

Python precommit failure looks like a flake

@TheNeuralBit
TheNeuralBit merged commit 2e72638 into apache:masterDec 21, 2021
@lukecwik

Copy link
Copy Markdown
Member

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?

Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

@aaltay

Copy link
Copy Markdown
Member

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?

Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

@lukecwik - should this be a jira tracked as part of https://issues.apache.org/jira/browse/BEAM-13430 ?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@dpcollins-google@TheNeuralBit@lukecwik@aaltay