Skip to content

HBASE-22322 Use special pause for CallQueueTooBigException - #195

Merged
Apache9 merged 1 commit into
apache:masterfrom
Apache9:HBASE-22322
Apr 30, 2019
Merged

HBASE-22322 Use special pause for CallQueueTooBigException#195
Apache9 merged 1 commit into
apache:masterfrom
Apache9:HBASE-22322

Conversation

@Apache9

Copy link
Copy Markdown
Contributor

Still have some problems on implementing UTs, will update the patch soon. And maybe we could change the config name? I think a more general reason to use a larger pause is that the region server is overloaded, CallQueueTooBigException is only one possible result of a overloaded region server. So maybe we could change the config and also the new methods in TableBuilder and AdminBuilder to something like pauseWhenOverloaded? What do you think @carp84 . Thanks.

@Apache9
Apache9 requested a review from carp84April 28, 2019 15:03
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
0reexec51Docker mode activated.
_ Prechecks _
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ master Compile Tests _
0mvndep30Maven dependency ordering for branch
+1mvninstall274master passed
+1compile96master passed
+1checkstyle122master passed
+1shadedjars294branch has no errors when building our shaded downstream artifacts.
+1findbugs296master passed
+1javadoc54master passed
_ Patch Compile Tests _
0mvndep15Maven dependency ordering for patch
+1mvninstall248the patch passed
+1compile76the patch passed
+1javac76the patch passed
+1checkstyle32hbase-client: The patch generated 0 new + 21 unchanged - 9 fixed = 21 total (was 30)
+1checkstyle72The patch passed checkstyle in hbase-server
+1whitespace0The patch has no whitespace issues.
+1shadedjars272patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck555Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
+1findbugs403the patch passed
+1javadoc55the patch passed
_ Other Tests _
+1unit190hbase-client in the patch passed.
+1unit12299hbase-server in the patch passed.
+1asflicense47The patch does not generate ASF License warnings.
15580
SubsystemReport/Notes
DockerClient=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/1/artifact/out/Dockerfile
GITHUB PR#195
Optional Testsdupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 1a45d0f6bf0b 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec 10 13:20:24 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/testptch/patchprocess/precommit/personality/provided.sh
git revisionmaster / 4477dd5
mavenversion: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java1.8.0_181
findbugsv3.1.11
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/1/testReport/
Max. process+thread count4735 (vs. ulimit of 10000)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/1/console
Powered byApache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@carp84

Copy link
Copy Markdown
Member

Maybe we could change the config name? I think a more general reason to use a larger pause is that the region server is overloaded, CallQueueTooBigException is only one possible result of an overloaded region server. So maybe we could change the config and also the new methods in TableBuilder and AdminBuilder to something like pauseWhenOverloaded?

Agreed. Actually the original config name proposed in HBASE-17114 is hbase.client.pause.special but was concerned in this comment thus changed to some more specific one to cqtbe.

Will be back and review the PR soon.

* times) are both limitations for retrying, we will stop retrying when we reach any of the
* limitations.
* @param timeout
* @param unit

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to add some javadoc to parameters here instead of removing? I mean after all we are changing this part, let's improve it.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not have javadoc for these parameters in both AsyncTableBuilder and TableBuilder, and I think the parameter name is good enough to tell users the meaning?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will generate checkstyle error by default although not a big deal.

/**
* Set timeout for each rpc request.
* @param timeout
* @param unit

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

* Set the base pause time for retrying. We use an exponential policy to generate sleep time when
* retrying.
* @param timeout
* @param unit

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

* Set the max retry times for an admin operation. Usually it is the max attempt times minus 1.
* Operation timeout and max attempt times(or max retry times) are both limitations for retrying,
* we will stop retrying when we reach any of the limitations.
* @param maxRetries

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

* Set the max attempt times for an admin operation. Usually it is the max retry times plus 1.
* Operation timeout and max attempt times(or max retry times) are both limitations for retrying,
* we will stop retrying when we reach any of the limitations.
* @param maxAttempts

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.


/**
* Set the number of retries that are allowed before we start to log.
* @param startLogErrorsCnt

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

@carp84carp84 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, just some minor comments that don't block the commit. And please make it compatible for old configuration if plan to change the config name.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec295Docker mode activated.
_ Prechecks _
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ master Compile Tests _
0mvndep26Maven dependency ordering for branch
+1mvninstall296master passed
+1compile99master passed
+1checkstyle125master passed
+1shadedjars332branch has no errors when building our shaded downstream artifacts.
+1findbugs364master passed
+1javadoc67master passed
_ Patch Compile Tests _
0mvndep17Maven dependency ordering for patch
+1mvninstall294the patch passed
+1compile98the patch passed
+1javac98the patch passed
+1checkstyle38hbase-client: The patch generated 0 new + 21 unchanged - 9 fixed = 21 total (was 30)
+1checkstyle77The patch passed checkstyle in hbase-server
+1whitespace0The patch has no whitespace issues.
+1shadedjars329patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck619Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
+1findbugs382the patch passed
+1javadoc66the patch passed
_ Other Tests _
+1unit222hbase-client in the patch passed.
-1unit15840hbase-server in the patch failed.
+1asflicense56The patch does not generate ASF License warnings.
19747
ReasonTests
Failed junit testshadoop.hbase.client.TestAsyncTableAdminApi
hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas
SubsystemReport/Notes
DockerClient=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/2/artifact/out/Dockerfile
GITHUB PR#195
Optional Testsdupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 6341fd104ff4 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/testptch/patchprocess/precommit/personality/provided.sh
git revisionmaster / 4477dd5
mavenversion: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java1.8.0_181
findbugsv3.1.11
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/2/testReport/
Max. process+thread count5123 (vs. ulimit of 10000)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-195/2/console
Powered byApache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@Apache9

Copy link
Copy Markdown
ContributorAuthor

Agreed. Actually the original config name proposed in HBASE-17114 is hbase.client.pause.special but was concerned in this comment thus changed to some more specific one to cqtbe.

OK, can do this in another issue...

@Apache9
Apache9 merged commit f9f6354 into apache:masterApr 30, 2019
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Apache9@Apache-HBase@carp84