Skip to content

HBASE-28595: client side fix for partial results when exception from scan RPC is lost - #5904

Closed
csringhofer wants to merge 1 commit into
apache:branch-2from
csringhofer:HBASE-28595-branch-2
Closed

HBASE-28595: client side fix for partial results when exception from scan RPC is lost#5904
csringhofer wants to merge 1 commit into
apache:branch-2from
csringhofer:HBASE-28595-branch-2

Conversation

@csringhofer

@csringhofercsringhofer commented May 15, 2024

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 42sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall2m 52sbranch-2 passed
+1 💚compile0m 50sbranch-2 passed
+1 💚checkstyle0m 20sbranch-2 passed
+1 💚spotless0m 45sbranch has no errors when running spotless:check.
+1 💚spotbugs0m 54sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 50sthe patch passed
+1 💚compile0m 48sthe patch passed
+1 💚javac0m 48sthe patch passed
-0 ⚠️checkstyle0m 19shbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 16sPatch does not cause any errors with Hadoop 2.10.2 or 3.3.6.
-1 ❌spotless0m 20spatch has 26 errors when running spotless:check, run spotless:apply to fix.
+1 💚spotbugs1m 0sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
22m 33s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux d6c334e9c8ae 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 44sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 34sbranch-2 passed
+1 💚compile0m 17sbranch-2 passed
+1 💚shadedjars5m 7sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 16sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 14sthe patch passed
+1 💚compile0m 17sthe patch passed
+1 💚javac0m 17sthe patch passed
+1 💚shadedjars5m 8spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 15sthe patch passed
_ Other Tests _
+1 💚unit8m 6shbase-client in the patch passed.
26m 6s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 02edfaaaf150 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/testReport/
Max. process+thread count354 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@csringhofer
csringhoferforce-pushed the HBASE-28595-branch-2 branch from 4c30a9d to 9b9c9fcCompareMay 15, 2024 16:34
@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 42sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 4sbranch-2 passed
+1 💚compile0m 24sbranch-2 passed
+1 💚shadedjars6m 54sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 40sthe patch passed
+1 💚compile0m 22sthe patch passed
+1 💚javac0m 22sthe patch passed
+1 💚shadedjars5m 56spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 17sthe patch passed
_ Other Tests _
+1 💚unit8m 19shbase-client in the patch passed.
32m 12s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7cc15c8befd5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/testReport/
Max. process+thread count386 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 41sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall2m 47sbranch-2 passed
+1 💚compile0m 48sbranch-2 passed
+1 💚checkstyle0m 19sbranch-2 passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
+1 💚spotbugs0m 50sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 45sthe patch passed
+1 💚compile0m 50sthe patch passed
+1 💚javac0m 50sthe patch passed
-0 ⚠️checkstyle0m 20shbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 13sPatch does not cause any errors with Hadoop 2.10.2 or 3.3.6.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
+1 💚spotbugs0m 58sthe patch passed
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
22m 44s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux e38c21dde0b4 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 41sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 12sbranch-2 passed
+1 💚compile0m 18sbranch-2 passed
+1 💚shadedjars5m 7sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 15sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 19sthe patch passed
+1 💚compile0m 18sthe patch passed
+1 💚javac0m 18sthe patch passed
+1 💚shadedjars5m 6spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 15sthe patch passed
_ Other Tests _
+1 💚unit8m 5shbase-client in the patch passed.
25m 50s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 8dfabc00f862 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/testReport/
Max. process+thread count356 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 15, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 43sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall2m 50sbranch-2 passed
+1 💚compile0m 20sbranch-2 passed
+1 💚shadedjars5m 53sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 17sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 57sthe patch passed
+1 💚compile0m 20sthe patch passed
+1 💚javac0m 20sthe patch passed
+1 💚shadedjars5m 52spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 17sthe patch passed
_ Other Tests _
+1 💚unit8m 22shbase-client in the patch passed.
29m 16s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5904
JIRA IssueHBASE-28595
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 04be089160cd 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 860ac60
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/testReport/
Max. process+thread count367 (vs. ulimit of 30000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5904/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

taklwu
taklwu approved these changes May 16, 2024
View reviewed changes

@taklwutaklwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, IMO this is a clean client side fix.

@Apache9 I recalled on the JIRA HBASE-28595, you mentioned it should be a server-side fix, should we have a separate JIRA for the server side change?

nit: @csringhofer do you think you can write a unit test for it?

Apache9
Apache9 reviewed May 16, 2024
View reviewed changes

@Apache9Apache9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that in impala, you want a client which can work with any HBase server versions but in HBase, this should be a server issue so we should fix it at server side.

@csringhofer

csringhofer commented May 16, 2024•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

I am working on creating unit tests for this - after that creating a server side fix doesn't seem hard.

I know that in impala, you want a client which can work with any HBase server versions but in HBase,

The main priority is to have something that is easy to backport to older branches and can't break anything.

@csringhofer

csringhofer commented May 16, 2024

Copy link
Copy Markdown
ContributorAuthor

Uploaded a server side fix for master:
#5910

Created a test that seemed consistent so far but had to modify RemoteWithExtrasException to make it work. I do not understand why this was needed.
the test is uploaded at csringhofer@17dd10e
Also, the test is likely to be sensitive to cpu speed and luck. To make it really reliable a way would be needed to inject delay to the processing of scan RPCs on the server.

@Apache9Apache9 closed this May 18, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 left review comments
@taklwutaklwutaklwu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@csringhofer@Apache-HBase@taklwu@Apache9