Skip to content

HBASE-25640 Support hbase rpc compression for remote rpc only - #3113

Open
huaxiangsun wants to merge 1 commit into
apache:masterfrom
huaxiangsun:master.HBASE-25640
Open

HBASE-25640 Support hbase rpc compression for remote rpc only#3113
huaxiangsun wants to merge 1 commit into
apache:masterfrom
huaxiangsun:master.HBASE-25640

Conversation

@huaxiangsun

Copy link
Copy Markdown
Contributor

I did manual test to make sure this change works, it is hard to put up a unittest for this change.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 58smaster passed
+1 💚compile0m 26smaster passed
+1 💚shadedjars7m 39sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 41sthe patch passed
+1 💚compile0m 26sthe patch passed
+1 💚javac0m 26sthe patch passed
+1 💚shadedjars7m 37spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 23sthe patch passed
_ Other Tests _
+1 💚unit1m 12shbase-client in the patch passed.
27m 28s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 2cc05d3d31b2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0242489
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count335 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 18sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 47smaster passed
+1 💚compile0m 31smaster passed
+1 💚shadedjars8m 34sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 42sthe patch passed
+1 💚compile0m 33sthe patch passed
+1 💚javac0m 33sthe patch passed
+1 💚shadedjars8m 40spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28sthe patch passed
_ Other Tests _
+1 💚unit1m 21shbase-client in the patch passed.
32m 44s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 187c557ce316 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0242489
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count298 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 19smaster passed
+1 💚compile0m 58smaster passed
+1 💚checkstyle0m 28smaster passed
+1 💚spotbugs1m 3smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 58sthe patch passed
+1 💚compile0m 53sthe patch passed
+1 💚javac0m 53sthe patch passed
-0 ⚠️checkstyle0m 26shbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 46sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs1m 12sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
42m 51s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3113
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 9c01c0b537d5 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0242489
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ndimidukndimiduk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some code style suggestions. I agree that a test would be helpful. Looking into our RpcConnection code, I don't have any strong suggestions

this.timeoutTimer = timeoutTimer;
this.codec = codec;
this.compressor = compressor;
if (compressor != null) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no need to check if compressor is null because you never read it, and subsequent code is apparently fine with a null value.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking if compressor is null is that it can avoid a dns resolution if compressor is null (no compression is configured), which is the common use case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, okay. A comment to this effect would help the casual reader.

this.compressor = compressor;
if (compressor != null) {
// Only enable compression for remote rpcs.
InetSocketAddress remoteAddr = Address.toSocketAddress(remoteId.getAddress());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this would be easier to read if you extract the isLocal(ConnectionId) logic into an external method. Then you can have a simpler conditional clause. Something like,

 this.compressor = isLocal(remoteId) ? null : compressor;

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 27sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 41smaster passed
+1 💚compile0m 31smaster passed
+1 💚shadedjars8m 14sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 29smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 26sthe patch passed
+1 💚compile0m 31sthe patch passed
+1 💚javac0m 31sthe patch passed
+1 💚shadedjars8m 10spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27sthe patch passed
_ Other Tests _
+1 💚unit1m 28shbase-client in the patch passed.
30m 29s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 661ec41fd966 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 9e47358
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count295 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 9smaster passed
+1 💚compile0m 59smaster passed
+1 💚checkstyle0m 29smaster passed
+1 💚spotbugs1m 3smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 48sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
-0 ⚠️checkstyle0m 26shbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 0sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs1m 13sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
40m 45s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3113
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 9d7c3f5502c2 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 9e47358
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 24sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall5m 3smaster passed
+1 💚compile0m 34smaster passed
+1 💚shadedjars12m 31sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 41sthe patch passed
+1 💚compile0m 37sthe patch passed
+1 💚javac0m 37sthe patch passed
+1 💚shadedjars11m 12spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30sthe patch passed
_ Other Tests _
+1 💚unit2m 0shbase-client in the patch passed.
41m 11s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 526da4420f05 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 9e47358
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count222 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 48sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 21smaster passed
+1 💚compile0m 28smaster passed
+1 💚shadedjars8m 19sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 4sthe patch passed
+1 💚compile0m 27sthe patch passed
+1 💚javac0m 27sthe patch passed
+1 💚shadedjars8m 48spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
-1 ❌unit1m 6shbase-client in the patch failed.
31m 15s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7113088aba15 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-client.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count339 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 49smaster passed
+1 💚compile0m 30smaster passed
+1 💚shadedjars8m 30sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 32sthe patch passed
+1 💚compile0m 32sthe patch passed
+1 💚javac0m 32sthe patch passed
+1 💚shadedjars8m 19spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
+1 💚unit1m 41shbase-client in the patch passed.
31m 15s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3113
Optional Testsjavac javadoc unit shadedjars compile
unameLinux e28e7026a3c9 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/testReport/
Max. process+thread count308 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 30sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall5m 41smaster passed
+1 💚compile1m 13smaster passed
+1 💚checkstyle0m 37smaster passed
+1 💚spotbugs1m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 18sthe patch passed
+1 💚compile1m 9sthe patch passed
+1 💚javac1m 9sthe patch passed
-0 ⚠️checkstyle0m 34shbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck22m 51sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs1m 17sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
51m 53s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3113
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux eac5e09bbe6c 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3113/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huaxiangsun@Apache-HBase@ndimiduk