Skip to content

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs - #3523

Closed
gjacoby126 wants to merge 1 commit into
apache:masterfrom
gjacoby126:HBASE-26115
Closed

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs#3523
gjacoby126 wants to merge 1 commit into
apache:masterfrom
gjacoby126:HBASE-26115

Conversation

@gjacoby126

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 48sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 17sMaven dependency ordering for branch
+1 💚mvninstall4m 44smaster passed
+1 💚compile5m 48smaster passed
+1 💚checkstyle2m 13smaster passed
+1 💚spotbugs4m 49smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall4m 33sthe patch passed
+1 💚compile5m 47sthe patch passed
+1 💚javac5m 47sthe patch passed
+1 💚checkstyle2m 29sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck21m 8sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs5m 22sthe patch passed
_ Other Tests _
+1 💚asflicense0m 44sThe patch does not generate ASF License warnings.
69m 53s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3523
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux c400644e6549 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d15f3cb
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 8sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for branch
+1 💚mvninstall4m 36smaster passed
+1 💚compile2m 29smaster passed
+1 💚shadedjars8m 43sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 48smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall4m 30sthe patch passed
+1 💚compile2m 38sthe patch passed
+1 💚javac2m 38sthe patch passed
+1 💚shadedjars8m 50spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 44sthe patch passed
_ Other Tests _
+1 💚unit2m 5shbase-common in the patch passed.
+1 💚unit0m 47shbase-zookeeper in the patch passed.
+1 💚unit146m 15shbase-server in the patch passed.
+1 💚unit1m 45shbase-testing-util in the patch passed.
190m 47s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 3a2d0e0bb829 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d15f3cb
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/testReport/
Max. process+thread count3866 (vs. ulimit of 30000)
modulesC: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for branch
+1 💚mvninstall4m 30smaster passed
+1 💚compile2m 16smaster passed
+1 💚shadedjars9m 18sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 31smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall4m 8sthe patch passed
+1 💚compile2m 7sthe patch passed
+1 💚javac2m 7sthe patch passed
+1 💚shadedjars9m 18spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 31sthe patch passed
_ Other Tests _
+1 💚unit2m 2shbase-common in the patch passed.
+1 💚unit0m 48shbase-zookeeper in the patch passed.
+1 💚unit252m 31shbase-server in the patch passed.
+1 💚unit2m 21shbase-testing-util in the patch passed.
297m 13s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6f3642ee8fe1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d15f3cb
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/testReport/
Max. process+thread count2741 (vs. ulimit of 30000)
modulesC: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

* Common helpers for testing HBase that do not depend on specific server/etc. things.
*/
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.PHOENIX)
@InterfaceAudience.LimitedPrivate({HBaseInterfaceAudience.COPROC,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, in general let's not do this.

Mark it as IA.LP("Phoenix") is already a compromise, let's not add more compromise here...

@gjacoby126gjacoby126Jul 23, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand what the compromise is here?

For example, I wear three different hats. I write code for HBase itself, for Phoenix, and for my dayjob. You seem to be arguing that the type of test infrastructure I require depends on what hat I'm wearing, not on the code I'm writing. Are you saying that server-side component tests are good when I wear the HBase hat, a tolerated compromise when I wear the Phoenix hat, and bad when I wear the dayjob hat? And that that would be the case _even for the same code _ depending on where I intended to push it when it's done?

In my view, HBase tests that just exercise the client API should use the TestingHBaseCluster. Same with Phoenix tests, or dayjob tests that only rely on the HBase client API. Tests that exercise server-side components that need the extra control over server-side state should be able to do so. That's true whether the components are housed in an HBase repo, or a Phoenix repo, or any other project, open source or not.

HBase gives a lot of ways to create server side code, and they all need to be tested. The IA should recognize that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a native English speaker so sorry I can not understand what you mean.

What I say compromise here, is in HBASE-13126, we want to mark HBTU as IA.Private, but consider the current situation, at least we need to mark it as IA.LP("Phoenix"), as we are sure that Phoenix needs to test something internal to HBase. This is a compromise.

But if you offer your help on improving the new testing clsuter interface, I think maybe we could finish the support for Phoenix before 3.0.0 is out, then probably we could mark HBTU as IA.Private in 3.0.0 finally.

Thanks.

*
* @return the utility running the minicluster
*/
HBaseTestingUtil getUtil();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a no no.

Let's find another way to only expose the necessary methods for testing coprocessors and other server side hooks such as tags. The point here is, HBaseTestingUtil is a class, not an interface, which makes very hard for the HBase developper to make it stable while keeping it clean.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Apache9 - thanks, that's a good point. In the next revision of this patch, I will create an interface for HBTU (and probably its superclasses) so this interface doesn't return a concrete class. I'm going to be on vacation for about a week but will take this up when I get back.

@gjacoby126

Copy link
Copy Markdown
ContributorAuthor

Closing for now as HBASE-26157 is an alternate solution to the same problem

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gjacoby126@Apache-HBase@Apache9