[BEAM-14053] [CdapIO] Add wrapper class for CDAP plugin by ktttnv · Pull Request #17150 · apache/beam

ktttnv · 2022-03-22T11:56:10Z

Warning: This PR contains changes from PR "[BEAM-14048] [CdapIO] Add ConfigWrapper for building CDAP PluginConfigs" and should be merged after it

Changes:

Added wrapper classes for source and sink CDAP plugins
Added builder for plugin wrapper class
Added possibility to provide Hadoop configuration

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

codecov · 2022-03-22T12:36:35Z

Codecov Report

Merging #17150 (94eca21) into master (a167424) will increase coverage by 0.16%.
The diff coverage is n/a.

❗ Current head 94eca21 differs from pull request most recent head 2766dca. Consider uploading reports for the commit 2766dca to get more accurate results

@@ Coverage Diff @@## master #17150 +/- ##
==========================================
+ Coverage 73.96% 74.12% +0.16% 
==========================================
Files 693 680 -13 Lines 91721 89653 -2068 ==========================================
- Hits 67838 66454 -1384 + Misses 22634 22096 -538 + Partials 1249 1103 -146

Flag	Coverage Δ
python	`83.71% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdks/go/pkg/beam/core/util/reflectx/types.go	`0.00% <0.00%> (-94.60%)`	⬇️
sdks/go/pkg/beam/core/util/reflectx/call.go	`0.00% <0.00%> (-89.75%)`	⬇️
sdks/go/pkg/beam/core/graph/coder/panes.go	`0.00% <0.00%> (-81.25%)`	⬇️
sdks/go/pkg/beam/core/util/reflectx/structs.go	`0.00% <0.00%> (-76.93%)`	⬇️
sdks/go/pkg/beam/core/runtime/graphx/cogbk.go	`0.00% <0.00%> (-72.73%)`	⬇️
sdks/go/pkg/beam/core/runtime/exec/discard.go	`0.00% <0.00%> (-71.43%)`	⬇️
sdks/go/pkg/beam/io/datastoreio/datastore.go	`16.03% <0.00%> (-51.78%)`	⬇️
sdks/go/pkg/beam/core/util/reflectx/functions.go	`54.54% <0.00%> (-27.28%)`	⬇️
sdks/go/pkg/beam/provision/provision.go	`27.50% <0.00%> (-20.00%)`	⬇️
sdks/go/pkg/beam/core/runtime/pipelinex/clone.go	`21.42% <0.00%> (-18.58%)`	⬇️
... and 69 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a167424...2766dca. Read the comment docs.

chamikaramj · 2022-03-23T16:43:40Z

Retest this please

…M-14048-cdap-config # Conflicts: # sdks/java/io/cdap/build.gradle

# Conflicts: # buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

# Conflicts: # buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy # sdks/java/io/cdap/build.gradle

# Conflicts: # sdks/java/io/cdap/build.gradle

elizaveta-lomteva · 2022-05-13T11:37:31Z

Run Java PreCommit

Amar3tto · 2022-05-16T08:06:27Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-16T12:01:57Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-17T09:52:56Z

@chamikaramj this PR is ready for review

bvolpato · 2022-05-18T16:12:27Z

+ for (Annotation annotation : getPluginClass().getDeclaredAnnotations()) {
+ if (annotation.annotationType().equals(io.cdap.cdap.api.annotation.Plugin.class)) {


Just a nitpick, but there is a method getDeclaredAnnotationsByType that should work as a shortcut.

Thank you, that's helpful. Done

elizaveta-lomteva · 2022-05-20T18:57:00Z

Run Java PreCommit

chamikaramj

Thanks. LGTM.

chamikaramj · 2022-05-23T01:28:41Z

Run Java PreCommit

Amar3tto · 2022-05-23T05:47:42Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-23T08:04:57Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-23T13:08:05Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-24T06:40:55Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-24T15:15:20Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-24T16:33:53Z

Run Java PreCommit

chamikaramj · 2022-05-24T18:18:19Z

Run Java PreCommit

elizaveta-lomteva · 2022-05-24T18:41:18Z

@chamikaramj PR is ready to be merged

Amar3ttoand others added 20 commits March 9, 2022 18:21

[BEAM-14048] Add ConfigWrapper for building CDAP PluginConfigs
78dd87b

[BEAM-14048] Fix checkstyle
30b9e38

[BEAM-14048] Fix warnings
9e5e6d6

[BEAM-14048] Fix warnings
2d414f4

[BEAM-14048] Fix warning
d35acb8

[BEAM-14048] Fix warning
b7e0fca

[BEAM-14048] Remove unused dependencies
3252395

[BEAM-14048] Add needed dependencies
f9e5675

[BEAM-14048] Fix spotless
65a5442

[BEAM-14048] Fix typo
00aef64

[BEAM-14048] Use fori instead of stream
d10b84b

[BEAM-14048] Suppress warning
2f9686d

[BEAM-14048] Add used undeclared artifacts
119d2cc

[BEAM-14048] Change dependencies to test
1f63d65

Add context.
6207224

Merge branch 'BEAM-14048-cdap-config' into BEAM-14081-context-wrapper
8b5aecb

Fix dependencies issue
1dbd4d6

Add null annotation
0777205

Added CDAP plugin wrapper
86acc42

[BEAM-14048] Refactoring
0645c7b

github-actionsBot added build io java labels Mar 22, 2022

igor.krasavinand others added 4 commits March 23, 2022 13:41

Add SuppressWarning.
7251307

Fix style.
dec0c3c

Determine dependencies.
4cacd1c

Merge branch 'apache:master' into BEAM-14048-cdap-config
44125f0

chamikaramj closed this Mar 23, 2022

Amar3ttoand others added 12 commits April 29, 2022 14:49

[BEAM-14053] Add avro dependency
400bcc7

[BEAM-14048] Use ServiceNow CDAP dependency from Maven central
f440da9

Merge remote-tracking branch 'origin/BEAM-14048-cdap-config' into BEA…
7c0028c
…M-14048-cdap-config # Conflicts: # sdks/java/io/cdap/build.gradle

[BEAM-14048] Set macroFields
9df1c75

Merge branch 'BEAM-14048-cdap-config' into BEAM-14081-context-wrapper
c5babc2
# Conflicts: # buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

Merge branch 'master' into BEAM-14081-context-wrapper
9701033
# Conflicts: # buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy # sdks/java/io/cdap/build.gradle

[BEAM-14081] Fix javadoc
9e31a72

Merge branch 'BEAM-14081-context-wrapper' into BEAM-14053-cdap-plugin
d236043
# Conflicts: # sdks/java/io/cdap/build.gradle

[BEAM-14053] Use ServiceNow dependency in tests
3f5008b

Merge remote-tracking branch 'origin/master' into BEAM-14053-cdap-plugin
94eca21

Merge branch 'master' into BEAM-14053-cdap-plugin
1883e11

Added hadoop_common dependency
73a1e55

bvolpato reviewed May 18, 2022
View reviewed changes

used getDeclaredAnnotationsByType method
2766dca

chamikaramj reviewed May 23, 2022
View reviewed changes

chamikaramj merged commit 7547cb0 into apache:masterMay 24, 2022

		for (Annotation annotation : getPluginClass().getDeclaredAnnotations()) {
		if (annotation.annotationType().equals(io.cdap.cdap.api.annotation.Plugin.class)) {

Uh oh!

Conversation

ktttnv commented Mar 22, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GitHub Actions Tests Status (on master branch)

Uh oh!

codecovBot commented Mar 22, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

chamikaramj commented Mar 23, 2022

Uh oh!

elizaveta-lomteva commented May 13, 2022

Uh oh!

Amar3tto commented May 16, 2022

Uh oh!

elizaveta-lomteva commented May 16, 2022

Uh oh!

elizaveta-lomteva commented May 17, 2022

Uh oh!

bvolpatoMay 18, 2022

Choose a reason for hiding this comment

Uh oh!

elizaveta-lomtevaMay 20, 2022

Choose a reason for hiding this comment

Uh oh!

elizaveta-lomteva commented May 20, 2022

Uh oh!

chamikaramj left a comment

Choose a reason for hiding this comment

Uh oh!

chamikaramj commented May 23, 2022

Uh oh!

Amar3tto commented May 23, 2022

Uh oh!

elizaveta-lomteva commented May 23, 2022

Uh oh!

elizaveta-lomteva commented May 23, 2022

Uh oh!

elizaveta-lomteva commented May 24, 2022

Uh oh!

elizaveta-lomteva commented May 24, 2022

Uh oh!

elizaveta-lomteva commented May 24, 2022

Uh oh!

chamikaramj commented May 24, 2022

Uh oh!

elizaveta-lomteva commented May 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ktttnv commented Mar 22, 2022•
edited
Loading

codecovBot commented Mar 22, 2022•
edited
Loading