Uh oh!
There was an error while loading. Please reload this page.
HBASE-23113: Improve and add additional Netty configuration for RPC. - #1440
Closed
markrmiller wants to merge 1 commit into
Closed
HBASE-23113: Improve and add additional Netty configuration for RPC.#1440markrmiller wants to merge 1 commit into
markrmiller wants to merge 1 commit into
Conversation
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
saintstack
left a comment
Contributor
There was a problem hiding this comment.
Patch looks good. Just adding extra configurable params w/ sensible looking defaults. Is the server tcp queue count low though?
| "hbase.ipc.server.bufferhighwatermark"; | ||
| protected static final boolean DEFAULT_SERVER_REUSEADDR = true; | ||
| protected static final int DEFAULT_SERVER_TCP_BACKLOG = 1024; |
Member
There was a problem hiding this comment.
Yeah, this lower end is much lower than the default.
privatestaticfinalintDEFAULT_LOW_WATER_MARK = 32 * 1024;
privatestaticfinalintDEFAULT_HIGH_WATER_MARK = 64 * 1024;
publicstaticfinalWriteBufferWaterMarkDEFAULT =
newWriteBufferWaterMark(DEFAULT_LOW_WATER_MARK, DEFAULT_HIGH_WATER_MARK, false);MemberAuthor
There was a problem hiding this comment.
The TCP_BACKLOG? I think the linux default is often 128 as mentioned in the JIRA? Raising it too high is also often a mistake, so I went with what the original author did. Have another suggestion?
MemberAuthor
There was a problem hiding this comment.
For the WATER_MARK settings, the previous pr is using:
protected static final int DEFAULT_SERVER_BUFFER_LOW_WATERMARK = 1024;
protected static final int DEFAULT_SERVER_BUFFER_HIGH_WATERMARK = 64 * 1024;
If the number of queued bytes goes above the high mark, then channel.isWritable will return false until the number of outstanding bytes falls below the low water mark.
Member
There was a problem hiding this comment.
The TCP_BACKLOG?
Sorry, no. I had this on the wrong line. I mean DEFAULT_SERVER_BUFFER_LOW_WATERMARK = 1024.
If the number of queued bytes goes above the high mark, then channel.isWritable will return false until the number of outstanding bytes falls below the low water mark.
Right. So I'm asking you (and I guess @SteNicholas as well): why do we need to flush out the buffer down almost completely before we continue receiving data? Is this assuming that HBase clients are generally writing large amounts of data, and this to avoid thrashing back and forth between threads as we fill, flush, fill, flush? A setting this low means we'd have to drain the buffer almost entirely before returning to the user thread.
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 6, 2020
🎊 +1 overall
This message was automatically generated. |
ndimiduk
reviewed
Apr 6, 2020
| protected final WriteBufferWaterMark writeBufferWaterMark; | ||
| protected static final String CLIENT_CONNECT_MAX_RETRIES = "hbase.ipc.client.connect.max.retries"; |
Member
There was a problem hiding this comment.
These first three are unused. Any anyway, the parent class manages reading these values from configuration.
MemberAuthor
There was a problem hiding this comment.
Copy paste error, should just be WATERMARK stuff.
| .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, rpcClient.connectTO) | ||
| .handler(new BufferCallBeforeInitHandler()).localAddress(rpcClient.localAddr) | ||
| .option(ChannelOption.WRITE_BUFFER_WATER_MARK, | ||
| ((NettyRpcClient) rpcClient).writeBufferWaterMark) |
MemberAuthor
There was a problem hiding this comment.
Good catch, no longer necessary, from an earlier iteration.
| protected static final String SERVER_TCP_BACKLOG = "hbase.ipc.server.tcpbacklog"; | ||
| protected static final String SERVER_TCP_REUSEADDR = "hbase.ipc.server.tcpreuseaddr"; | ||
| protected static final String SERVER_TCP_NODELAY = "hbase.ipc.server.tcpnodelay"; |
Member
There was a problem hiding this comment.
NODELAY and KEEPALIVE are unused. Also managed by the parent class.
MemberAuthor
There was a problem hiding this comment.
Same copy paste mistake as above - previous PR had this all in the base class.
| "hbase.ipc.server.bufferhighwatermark"; | ||
| protected static final boolean DEFAULT_SERVER_REUSEADDR = true; | ||
| protected static final int DEFAULT_SERVER_TCP_BACKLOG = 1024; |
Member
There was a problem hiding this comment.
Yeah, this lower end is much lower than the default.
privatestaticfinalintDEFAULT_LOW_WATER_MARK = 32 * 1024;
privatestaticfinalintDEFAULT_HIGH_WATER_MARK = 64 * 1024;
publicstaticfinalWriteBufferWaterMarkDEFAULT =
newWriteBufferWaterMark(DEFAULT_LOW_WATER_MARK, DEFAULT_HIGH_WATER_MARK, false);ndimiduk
commented
Apr 8, 2020
Member
Looks like this one supersedes #679 . |
Apache-HBase
commented
Apr 11, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 11, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 11, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 11, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 12, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 12, 2020
💔 -1 overall
This message was automatically generated. |
markrmiller
commented
Apr 12, 2020
MemberAuthor
Thanks, that took the settings as given in the original pr for master it looks. |
ndimiduk
commented
Apr 14, 2020
Member
Let's get the change to expose these configs into branch-2.3. We'll leave the default settings as they are until we have some proof that new defaults are doing some good. |
Apache-HBase
commented
Apr 14, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 15, 2020
💔 -1 overall
This message was automatically generated. |
saintstack
commented
Apr 15, 2020
Contributor
Failures look related. Complaint about netty on construction of RegionServer. |
Apache-HBase
commented
Apr 15, 2020
💔 -1 overall
This message was automatically generated. |
ndimiduk
commented
Apr 16, 2020
Member
@markrmiller you see the comment above? |
markrmiller
commented
Apr 20, 2020
MemberAuthor
I'll take a look and update this in a bit. |
saintstack
commented
May 4, 2020
Contributor
@markrmiller Any progress here boss? |
Apache-HBase
commented
May 22, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
May 22, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
May 22, 2020
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
May 28, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
May 28, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
May 28, 2020
🎊 +1 overall
This message was automatically generated. |
saintstack
commented
Oct 6, 2020
Contributor
Abandoned PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.