Skip to content

HBASE-24425 Run hbck_chore_run and catalogjanitor_run on draw of 'HBC… - #1771

Merged
saintstack merged 1 commit into
apache:branch-2from
saintstack:HBASE-24425
May 27, 2020
Merged

HBASE-24425 Run hbck_chore_run and catalogjanitor_run on draw of 'HBC…#1771
saintstack merged 1 commit into
apache:branch-2from
saintstack:HBASE-24425

Conversation

@saintstack

Copy link
Copy Markdown
Contributor

…K Report' page

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 28sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
_ Patch Compile Tests _
-0 ⚠️whitespace0m 0sThe patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
2m 0s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1771
Optional Testsdupname asflicense
unameLinux 128a9edf8c6d 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 0392977
whitespacehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-general-check/output/whitespace-eol.txt
Max. process+thread count42 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 17sbranch-2 passed
-0 ⚠️javadoc0m 41shbase-server in branch-2 failed.
_ Patch Compile Tests _
+1 💚mvninstall3m 59sthe patch passed
-0 ⚠️javadoc0m 39shbase-server in the patch failed.
_ Other Tests _
+1 💚unit120m 33shbase-server in the patch passed.
132m 56s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1771
Optional Testsjavac javadoc unit
unameLinux 9521dab4ac3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 0392977
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/testReport/
Max. process+thread count4546 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 34sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 41sbranch-2 passed
+1 💚javadoc0m 38sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 12sthe patch passed
+1 💚javadoc0m 34sthe patch passed
_ Other Tests _
+1 💚unit130m 56shbase-server in the patch passed.
141m 47s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#1771
Optional Testsjavac javadoc unit
unameLinux a91b4d6eefe5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 0392977
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/testReport/
Max. process+thread count3881 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@HorizonNetHorizonNet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. branch-2 only?

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left few comments. Looks good.

<%@ page import="org.apache.hadoop.hbase.util.Threads" %>
<%
final String cacheParameterValue = request.getParameter("cache");
boolean cache = Boolean.valueOf(cacheParameterValue);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Since we need boolean, we can directly use: Boolean.parseBoolean(cacheParameterValue)

if (!cache) {
// Run the two reporters inline w/ drawing of the page. If exception, will show in page draw.
master.getMasterRpcServices().runHbckChore(null, null);
master.getMasterRpcServices().runCatalogScan(null, null);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MasterRpcServices throw ServiceException for both runHbckChore and runCatalogScan. Maybe we can handle it and print some nice error message?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Was thinking about it. Though the rare SE not the end of the world. Added something at your prompting.... it looks like this when generated...

 if (!Boolean.parseBoolean(cacheParameterValue)) {
// Run the two reporters inline w/ drawing of the page. If exception, will show in page draw.
try {
master.getMasterRpcServices().runHbckChore(null, null);
} catch (org.apache.hbase.thirdparty.com.google.protobuf.ServiceException se) {
out.write("Failed generating a new hbck_chore report; using cache; try again or run hbck_chore_run in the shell: " + se.getMessage() + "\n");
}
try {
master.getMasterRpcServices().runCatalogScan(null, null);
} catch (org.apache.hbase.thirdparty.com.google.protobuf.ServiceException se) {
out.write("Failed generating a new catalogjanitor report; using cache; try again or run catalogjanitor_run in the shell: " + se.getMessage() + "\n");
}
}

<%@ page import="org.apache.hadoop.hbase.util.Pair" %>
<%@ page import="org.apache.hadoop.hbase.master.CatalogJanitor" %>
<%@ page import="org.apache.hadoop.hbase.master.CatalogJanitor.Report" %>
<%@ page import="org.apache.hadoop.hbase.util.Threads" %>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not being used it seems.

@virajjasani

Copy link
Copy Markdown
Contributor

LGTM. branch-2 only?

Looks like a clean apply to master and branch-2.3 👍

@saintstack

Copy link
Copy Markdown
ContributorAuthor

Address @virajjasani suggestions.

@HorizonNet I pushed against branch-2 since that is what I know and its stable.... will forward port when all is good.

Thanks for the reviews.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 34sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
_ Patch Compile Tests _
-0 ⚠️whitespace0m 0sThe patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
2m 3s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1771
Optional Testsdupname asflicense
unameLinux eb69aa032478 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3ef44a4
whitespacehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-general-check/output/whitespace-eol.txt
Max. process+thread count43 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 45sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall5m 35sbranch-2 passed
-0 ⚠️javadoc0m 49shbase-server in branch-2 failed.
_ Patch Compile Tests _
+1 💚mvninstall5m 18sthe patch passed
-0 ⚠️javadoc0m 49shbase-server in the patch failed.
_ Other Tests _
+1 💚unit135m 45shbase-server in the patch passed.
151m 23s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1771
Optional Testsjavac javadoc unit
unameLinux d962d21e4272 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3ef44a4
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/testReport/
Max. process+thread count3537 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 25sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 57sbranch-2 passed
+1 💚javadoc0m 40sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 38sthe patch passed
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
+1 💚unit211m 50shbase-server in the patch passed.
225m 13s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#1771
Optional Testsjavac javadoc unit
unameLinux aafe54afd2ad 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 3ef44a4
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/testReport/
Max. process+thread count2602 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@saintstack
saintstack merged commit f8a5cc9 into apache:branch-2May 27, 2020
saintstack added a commit that referenced this pull request May 27, 2020
…K Report' page (#1771)
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
saintstack added a commit that referenced this pull request May 27, 2020
…K Report' page (#1771)
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…K Report' page (apache#1771)
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@saintstack@Apache-HBase@virajjasani@HorizonNet@clarax@huaxiangsun