HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily by maobaolong · Pull Request #1369 · apache/ozone

maobaolong · 2020-09-01T00:37:52Z

What changes were proposed in this pull request?

Provide a way to get the default value and key of java-based-configuration easily.

What is the link to the Apache JIRA

HDDS-4104

How was this patch tested?

This PR with the related unit tests.

…ed-configuration easily

maobaolong · 2020-09-01T08:44:12Z

@elek Thanks to create the Java-based-Configuration framework, it is better than the hadoop configuration, but sometime, I want to get the default value or key name for a config variable, so I submit this PR, hope this can supply a way to use Java-based-Configuration framework conveniently.

adoroszlai

Thanks @maobaolong for working on this. I would like to propose some minor changes and further tests.

Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>

maobaolong · 2020-09-02T16:23:20Z

@adoroszlai Thanks for your review and the good suggestion, I push new commits and addressed the comments inline.
PTAL.

adoroszlai

Thanks @maobaolong for updating the patch, LGTM. Can you check if change in hadoop-ozone/dev-support/intellij/ozone-site.xml is intentional?

codecov-commenter · 2020-09-02T17:49:44Z

Codecov Report

Merging #1369 into master will decrease coverage by 0.19%.
The diff coverage is 100.00%.

@@ Coverage Diff @@## master #1369 +/- ##
============================================
- Coverage 75.32% 75.13% -0.20% + Complexity 10502 10501 -1 
============================================
Files 990 990 Lines 50757 50867 +110 Branches 4951 4959 +8 ============================================
- Hits 38233 38219 -14 - Misses 10158 10258 +100 - Partials 2366 2390 +24

Impacted Files	Coverage Δ	Complexity Δ
.../hadoop/hdds/conf/ConfigurationReflectionUtil.java	`80.62% <100.00%> (+4.20%)`	`40.00 <11.00> (+11.00)`
...hdds/scm/container/CloseContainerEventHandler.java	`72.41% <0.00%> (-17.25%)`	`6.00% <0.00%> (ø%)`
.../hadoop/ozone/recon/scm/ReconContainerManager.java	`83.05% <0.00%> (-13.56%)`	`13.00% <0.00%> (-1.00%)`
...ache/hadoop/ozone/om/codec/S3SecretValueCodec.java	`90.90% <0.00%> (-9.10%)`	`3.00% <0.00%> (-1.00%)`
...on/scm/ReconIncrementalContainerReportHandler.java	`60.00% <0.00%> (-7.50%)`	`4.00% <0.00%> (ø%)`
...va/org/apache/hadoop/hdds/utils/db/RDBMetrics.java	`92.85% <0.00%> (-7.15%)`	`14.00% <0.00%> (-1.00%)`
...doop/hdds/scm/container/ContainerStateManager.java	`81.67% <0.00%> (-6.88%)`	`32.00% <0.00%> (-3.00%)`
...rg/apache/hadoop/hdds/scm/server/SCMCertStore.java	`38.23% <0.00%> (-5.89%)`	`5.00% <0.00%> (-1.00%)`
.../org/apache/hadoop/hdds/scm/XceiverClientGrpc.java	`68.12% <0.00%> (-5.82%)`	`38.00% <0.00%> (ø%)`
...p/ozone/security/OzoneDelegationTokenSelector.java	`72.22% <0.00%> (-5.56%)`	`7.00% <0.00%> (-1.00%)`
... and 33 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d34ab29...67c37eb. Read the comment docs.

maobaolong · 2020-09-02T23:32:47Z

@adoroszlai PTAL.

adoroszlai

Thanks @maobaolong for removing unnecessary changes.

…onfigurationReflectionUtil.java Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>

maobaolong · 2020-09-03T16:00:23Z

@adoroszlai Thanks for your help during the reviewing.

maobaolong · 2020-09-04T02:57:09Z

@adoroszlai Thanks to trigger the CI check for me.

maobaolong · 2020-09-15T00:57:44Z

@elek Please take a look at this PR, thanks.

adoroszlai · 2020-09-17T09:25:38Z

Thanks @maobaolong for the contribution. Merged to master.

maobaolong · 2020-09-17T11:10:51Z

@adoroszlai Thanks you merge this PR.

* master: (47 commits) HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily (apache#1369) HDDS-4250. Fix wrong logger name (apache#1429) HDDS-4244. Container deleted wrong replica cause mis-replicated. (apache#1423) HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key. (apache#1296) HDDS-4210. ResolveBucket during checkAcls fails. (apache#1398) HDDS-4075. Retry request on different OM on AccessControlException (apache#1303) HDDS-4166. Documentation index page redirects to the wrong address (apache#1372) HDDS-4039. Reduce the number of fields in hdds.proto to improve performance (apache#1289) HDDS-4155. Directory and filename can end up with same name in a path. (apache#1361) HDDS-3927. Rename Ozone OM,DN,SCM runtime options to conform to naming conventions (apache#1401) HDDS-4119. Improve performance of the BufferPool management of Ozone client (apache#1336) HDDS-4217.Remove test TestOzoneContainerRatis (apache#1408) HDDS-4218.Remove test TestRatisManager (apache#1409) HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE. (apache#1337) HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log. (apache#1413) HDDS-4196. Add an endpoint in Recon to query Prometheus (apache#1390) HDDS-4211. [OFS] Better owner and group display for listing Ozone volumes and buckets (apache#1397) HDDS-4150. recon.api.TestEndpoints test is flaky (apache#1396) HDDS-4170 - Fix typo in method description. (apache#1406) HDDS-4064. Show container verbose info with verbose option (apache#1290) ...

…ponse * HDDS-4122-quota-attempt2: (51 commits) Remove redundant check status calls in children of AbstractOMOpenKeyDeleteRequest Remove unused inports and fix super constructor calls Move common volume byte usage update code to AbstractOMKeyDeleteResponse Add volume quota update to open key delete response, and group duplicate code HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily (apache#1369) HDDS-4250. Fix wrong logger name (apache#1429) HDDS-4244. Container deleted wrong replica cause mis-replicated. (apache#1423) HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key. (apache#1296) HDDS-4210. ResolveBucket during checkAcls fails. (apache#1398) HDDS-4075. Retry request on different OM on AccessControlException (apache#1303) HDDS-4166. Documentation index page redirects to the wrong address (apache#1372) HDDS-4039. Reduce the number of fields in hdds.proto to improve performance (apache#1289) HDDS-4155. Directory and filename can end up with same name in a path. (apache#1361) HDDS-3927. Rename Ozone OM,DN,SCM runtime options to conform to naming conventions (apache#1401) HDDS-4119. Improve performance of the BufferPool management of Ozone client (apache#1336) HDDS-4217.Remove test TestOzoneContainerRatis (apache#1408) HDDS-4218.Remove test TestRatisManager (apache#1409) HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE. (apache#1337) HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log. (apache#1413) HDDS-4196. Add an endpoint in Recon to query Prometheus (apache#1390) ...

HDDS-4104. Provide a way to get the default value and key of java-bas…
1fc00a3
…ed-configuration easily

adoroszlai requested changes Sep 1, 2020
View reviewed changes

Apply suggestions from code review
747c925
Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>

maobaolong force-pushed the HDDS-4104 branch 3 times, most recently from ac87b28 to 957cb5fCompare September 2, 2020 16:21

maobaolong requested a review from adoroszlai September 2, 2020 16:23

maobaolong added 2 commits September 3, 2020 00:23

add tests.
957cb5f

Fix style
67c37eb

adoroszlai reviewed Sep 2, 2020
View reviewed changes

Comment threadhadoop-ozone/dev-support/intellij/ozone-site.xml Outdated

maobaolong requested a review from adoroszlai September 2, 2020 23:32

revert ozone-site.xml and rename testcase.
e409d54

maobaolong closed this Sep 3, 2020

maobaolong reopened this Sep 3, 2020

maobaolong closed this Sep 3, 2020

maobaolong reopened this Sep 3, 2020

adoroszlai approved these changes Sep 3, 2020
View reviewed changes

adoroszlai reviewed Sep 3, 2020
View reviewed changes

Comment threadhadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java Outdated

Update hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/C…
a497618
…onfigurationReflectionUtil.java Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>

trigger new CI check
dc210f5

adoroszlai requested a review from elek September 4, 2020 06:50

adoroszlai merged commit 079ee7f into apache:masterSep 17, 2020

Uh oh!

Conversation

maobaolong commented Sep 1, 2020

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

maobaolong commented Sep 1, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maobaolong commented Sep 2, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov-commenter commented Sep 2, 2020

Codecov Report

Uh oh!

maobaolong commented Sep 2, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maobaolong commented Sep 3, 2020

Uh oh!

maobaolong commented Sep 4, 2020

Uh oh!

maobaolong commented Sep 15, 2020

Uh oh!

adoroszlai commented Sep 17, 2020

Uh oh!

maobaolong commented Sep 17, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants