Skip to content

HBASE-28389 HBase backup yarn queue parameter ignored - #6130

Merged
2005hithlj merged 1 commit into
apache:branch-2from
2005hithlj:HBASE-28389-branch-2
Jul 31, 2024
Merged

HBASE-28389 HBase backup yarn queue parameter ignored#6130
2005hithlj merged 1 commit into
apache:branch-2from
2005hithlj:HBASE-28389-branch-2

Conversation

@2005hithlj

@2005hithlj2005hithlj commented Jul 30, 2024

Copy link
Copy Markdown
Contributor

@2005hithlj

2005hithlj commented Jul 30, 2024

Copy link
Copy Markdown
ContributorAuthor

@DieterDP-ng@Apache9 Could you take a look? Thanks.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 48sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 1sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ branch-2 Compile Tests _
+1 💚mvninstall3m 5sbranch-2 passed
+1 💚compile0m 30sbranch-2 passed
+1 💚checkstyle0m 12sbranch-2 passed
+1 💚spotbugs0m 34sbranch-2 passed
+1 💚spotless0m 46sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 58sthe patch passed
+1 💚compile0m 27sthe patch passed
+1 💚javac0m 27sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 12sthe patch passed
+1 💚spotbugs0m 40sthe patch passed
+1 💚hadoopcheck15m 39sPatch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚spotless0m 44spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
28m 30s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6130
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux be2d5f8823d2 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / a5cf897
Default JavaEclipse Adoptium-11.0.23+9
Max. process+thread count83 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 42sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 38sbranch-2 passed
+1 💚compile0m 19sbranch-2 passed
+1 💚javadoc0m 15sbranch-2 passed
+1 💚shadedjars4m 59sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 30sthe patch passed
+1 💚compile0m 18sthe patch passed
+1 💚javac0m 18sthe patch passed
+1 💚javadoc0m 14sthe patch passed
+1 💚shadedjars5m 1spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit11m 55shbase-backup in the patch passed.
30m 2s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux d6bfaca083af 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / a5cf897
Default JavaTemurin-1.8.0_412-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count3410 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 20sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 8sbranch-2 passed
+1 💚compile0m 22sbranch-2 passed
+1 💚javadoc0m 17sbranch-2 passed
+1 💚shadedjars5m 30sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 58sthe patch passed
+1 💚compile0m 21sthe patch passed
+1 💚javac0m 21sthe patch passed
+1 💚javadoc0m 15sthe patch passed
+1 💚shadedjars5m 36spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit10m 40shbase-backup in the patch passed.
30m 35s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux e4c3fd1a7bb8 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / a5cf897
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count3182 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 43sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 10sbranch-2 passed
+1 💚compile0m 18sbranch-2 passed
+1 💚javadoc0m 14sbranch-2 passed
+1 💚shadedjars5m 56sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 57sthe patch passed
+1 💚compile0m 19sthe patch passed
+1 💚javac0m 19sthe patch passed
+1 💚javadoc0m 14sthe patch passed
+1 💚shadedjars5m 58spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit11m 27shbase-backup in the patch passed.
32m 26s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 7f39f84c6acc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / a5cf897
Default JavaEclipse Adoptium-11.0.23+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count3710 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

DieterDP-ng
DieterDP-ng approved these changes Jul 30, 2024
View reviewed changes

@DieterDP-ngDieterDP-ng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for solving this.

Comment threadhbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java Outdated
@@ -335,8 +335,8 @@ public void execute() throws IOException {

if (cmdline.hasOption(OPTION_YARN_QUEUE_NAME)) {

@DieterDP-ngDieterDP-ngJul 30, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest moving this block to before the super.execute(); that creates the connection. That way, you can directly update the conf using getConf(). Won't have any change in outcome, but feels a bit simpler.

@2005hithlj2005hithljJul 30, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DieterDP-ng Thanks for your review. I have revised it. could you take a look? Thanks.

@DieterDP-ngDieterDP-ngJul 30, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, no further remarks from me.

@2005hithlj
2005hithljforce-pushed the HBASE-28389-branch-2 branch from a5cf897 to 3bbd7b0CompareJuly 30, 2024 09:13
@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 42sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ branch-2 Compile Tests _
+1 💚mvninstall3m 9sbranch-2 passed
+1 💚compile0m 30sbranch-2 passed
+1 💚checkstyle0m 13sbranch-2 passed
+1 💚spotbugs0m 34sbranch-2 passed
+1 💚spotless0m 47sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 57sthe patch passed
+1 💚compile0m 27sthe patch passed
+1 💚javac0m 27sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 11sthe patch passed
+1 💚spotbugs0m 39sthe patch passed
+1 💚hadoopcheck15m 37sPatch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚spotless0m 45spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
28m 24s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6130
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 732c11e0be32 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3bbd7b0
Default JavaEclipse Adoptium-11.0.23+9
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 46sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 25sbranch-2 passed
+1 💚compile0m 18sbranch-2 passed
+1 💚javadoc0m 14sbranch-2 passed
+1 💚shadedjars5m 1sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 26sthe patch passed
+1 💚compile0m 19sthe patch passed
+1 💚javac0m 19sthe patch passed
+1 💚javadoc0m 14sthe patch passed
+1 💚shadedjars4m 57spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit11m 44shbase-backup in the patch passed.
29m 36s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 4e08f065b2b5 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3bbd7b0
Default JavaTemurin-1.8.0_412-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count2975 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 42sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 54sbranch-2 passed
+1 💚compile0m 18sbranch-2 passed
+1 💚javadoc0m 14sbranch-2 passed
+1 💚shadedjars5m 53sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 55sthe patch passed
+1 💚compile0m 18sthe patch passed
+1 💚javac0m 18sthe patch passed
+1 💚javadoc0m 13sthe patch passed
+1 💚shadedjars5m 56spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit11m 22shbase-backup in the patch passed.
31m 58s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 983f332cb4a7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3bbd7b0
Default JavaEclipse Adoptium-11.0.23+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count3821 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 30, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 20sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 22sbranch-2 passed
+1 💚compile0m 22sbranch-2 passed
+1 💚javadoc0m 16sbranch-2 passed
+1 💚shadedjars7m 3sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall4m 20sthe patch passed
+1 💚compile0m 30sthe patch passed
+1 💚javac0m 30sthe patch passed
+1 💚javadoc0m 21sthe patch passed
+1 💚shadedjars7m 14spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit13m 43shbase-backup in the patch passed.
38m 42s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6130
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 24d0a658c229 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3bbd7b0
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count3185 (vs. ulimit of 30000)
modulesC: hbase-backup U: hbase-backup
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@2005hithlj
2005hithlj merged commit 68045db into apache:branch-2Jul 31, 2024
2005hithlj added a commit that referenced this pull request Jul 31, 2024
@2005hithlj
HBASE-28389HBase backup yarn queue parameter ignored (#6130)
2747320
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com>
(cherry picked from commit 68045db)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 approved these changes
+1 more reviewer
@DieterDP-ngDieterDP-ngDieterDP-ng approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@2005hithlj@Apache-HBase@Apache9@DieterDP-ng