Skip to content

HBASE-25830 HBaseCluster support CompactionServer for UTs (addendum) - #3464

Merged
nyl3532016 merged 2 commits into
apache:HBASE-25714from
nyl3532016:HBASE-25830-2
Jul 14, 2021
Merged

HBASE-25830 HBaseCluster support CompactionServer for UTs (addendum)#3464
nyl3532016 merged 2 commits into
apache:HBASE-25714from
nyl3532016:HBASE-25830-2

Conversation

@nyl3532016

@nyl3532016nyl3532016 commented Jul 7, 2021

Copy link
Copy Markdown
Contributor

Support create/shutdown compactionserver method in HBaseMinicluster
Fix weakness of TestCompactionServer

@Apache-HBase

Apache-HBase commented Jul 7, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 30sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ HBASE-25714 Compile Tests _
+1 💚mvninstall4m 0sHBASE-25714 passed
+1 💚compile3m 11sHBASE-25714 passed
+1 💚checkstyle1m 6sHBASE-25714 passed
+1 💚spotbugs2m 5sHBASE-25714 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 36sthe patch passed
+1 💚compile3m 11sthe patch passed
+1 💚javac3m 11sthe patch passed
-0 ⚠️checkstyle1m 4shbase-server: The patch generated 4 new + 58 unchanged - 1 fixed = 62 total (was 59)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck18m 4sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 15sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
47m 9s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3464
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 6c95bd1e78b8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionHBASE-25714 / 7be5879
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 7, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚mvninstall4m 21sHBASE-25714 passed
+1 💚compile1m 4sHBASE-25714 passed
+1 💚shadedjars8m 29sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sHBASE-25714 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
+1 💚shadedjars8m 20spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
+1 💚unit215m 2shbase-server in the patch passed.
246m 40s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3464
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6a355265d23f 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionHBASE-25714 / 7be5879
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/testReport/
Max. process+thread count3350 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 7, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 30sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ HBASE-25714 Compile Tests _
+1 💚mvninstall4m 0sHBASE-25714 passed
+1 💚compile3m 9sHBASE-25714 passed
+1 💚checkstyle1m 7sHBASE-25714 passed
+1 💚spotbugs2m 7sHBASE-25714 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 36sthe patch passed
+1 💚compile3m 13sthe patch passed
+1 💚javac3m 13sthe patch passed
-0 ⚠️checkstyle1m 5shbase-server: The patch generated 4 new + 58 unchanged - 1 fixed = 62 total (was 59)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck18m 9sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 19sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
47m 19s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3464
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux f048d50a3387 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionHBASE-25714 / 7be5879
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 7, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚mvninstall3m 56sHBASE-25714 passed
+1 💚compile1m 3sHBASE-25714 passed
+1 💚shadedjars8m 27sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sHBASE-25714 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 6sthe patch passed
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
+1 💚shadedjars8m 31spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
+1 💚unit214m 23shbase-server in the patch passed.
245m 48s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3464
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 600de0d519a1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionHBASE-25714 / 7be5879
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/testReport/
Max. process+thread count3645 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@nyl3532016
nyl3532016 requested a review from Apache9July 8, 2021 01:52
Apache9
Apache9 reviewed Jul 11, 2021
View reviewed changes
Comment threadhbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java Outdated
/**
* Starts a compaction server thread running
*
* @throws IOException

@Apache9Apache9Jul 11, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove empty throws doc

Comment threadhbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java Outdated
/**
* Wait for the specified compaction server to stop. Removes this thread from list
* of running threads.
* @param serverNumber

@Apache9Apache9Jul 11, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: remove empty param doc

Comment threadhbase-server/src/test/java/org/apache/hadoop/hbase/compactionserver/TestCompactionServer.java Outdated
TEST_UTIL.compact(TABLENAME, false);
TEST_UTIL.waitFor(6000, () -> COMPACTION_SERVER.requestCount.sum() > 0);
long requestCount = COMPACTION_SERVER.requestCount.sum();
Thread.sleep(1000);

@Apache9Apache9Jul 11, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind explaining a bit here?

@nyl3532016nyl3532016Jul 11, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK,this sleep is no need. My Initially thought is compact request is async, this sleep will make the check below effective, After view code I find the increment of request count on compaction server is sync in compaction request

@Apache-HBase

Apache-HBase commented Jul 11, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 9sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚mvninstall4m 10sHBASE-25714 passed
+1 💚compile1m 3sHBASE-25714 passed
+1 💚shadedjars8m 28sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sHBASE-25714 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 7sthe patch passed
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
+1 💚shadedjars8m 25spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
+1 💚unit214m 33shbase-server in the patch passed.
245m 57s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3464
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 0c298f49e568 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionHBASE-25714 / 7be5879
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/testReport/
Max. process+thread count3415 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@nyl3532016
nyl3532016 requested a review from Apache9July 14, 2021 10:32
@nyl3532016
nyl3532016 merged commit 27ec80d into apache:HBASE-25714Jul 14, 2021
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nyl3532016@Apache-HBase@Apache9