Skip to content

HBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.nextRaw contract - #5304

Merged
bbeaudreault merged 1 commit into
apache:masterfrom
hgromer:HBASE-26630
Jun 23, 2023
Merged

HBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.nextRaw contract#5304
bbeaudreault merged 1 commit into
apache:masterfrom
hgromer:HBASE-26630

Conversation

@hgromer

@hgromerhgromer commented Jun 22, 2023•
edited by bbeaudreault
Loading

Copy link
Copy Markdown
Contributor

@Apache-HBase

Apache-HBase commented Jun 22, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
-1 ❌mvninstall0m 55sroot in master failed.
-1 ❌compile0m 14shbase-server in master failed.
+1 💚shadedjars4m 56sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 7shbase-server in master failed.
_ Patch Compile Tests _
+1 💚mvninstall2m 14sthe patch passed
+1 💚compile0m 36sthe patch passed
+1 💚javac0m 36sthe patch passed
+1 💚shadedjars4m 53spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 23shbase-server generated 23 new + 0 unchanged - 0 fixed = 23 total (was 0)
_ Other Tests _
-1 ❌unit9m 56shbase-server in the patch failed.
26m 4s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7e861f27a0b7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaTemurin-1.8.0_352-b08
mvninstallhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/branch-mvninstall-root.txt
compilehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/branch-compile-hbase-server.txt
javadochttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/testReport/
Max. process+thread count1778 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jun 22, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 31smaster passed
+1 💚compile2m 35smaster passed
+1 💚checkstyle0m 34smaster passed
+1 💚spotless0m 41sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 24smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 29sthe patch passed
+1 💚compile2m 23sthe patch passed
+1 💚javac2m 23sthe patch passed
+1 💚checkstyle0m 30sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 32sPatch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚spotless0m 39spatch has no errors when running spotless:check.
+1 💚spotbugs1m 28sthe patch passed
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
32m 18s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 52ddad65a9f4 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count79 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jun 22, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 49sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 52smaster passed
+1 💚compile0m 46smaster passed
+1 💚shadedjars4m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 42sthe patch passed
+1 💚compile0m 45sthe patch passed
+1 💚javac0m 45sthe patch passed
+1 💚shadedjars4m 34spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
-1 ❌unit218m 41shbase-server in the patch failed.
240m 46s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a6a3cbab711c 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/testReport/
Max. process+thread count4659 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jun 23, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 33smaster passed
+1 💚compile2m 27smaster passed
+1 💚checkstyle0m 33smaster passed
+1 💚spotless0m 40sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 20smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 30sthe patch passed
+1 💚compile2m 26sthe patch passed
+1 💚javac2m 26sthe patch passed
+1 💚checkstyle0m 34sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 24sPatch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚spotless0m 39spatch has no errors when running spotless:check.
+1 💚spotbugs1m 31sthe patch passed
_ Other Tests _
+1 💚asflicense0m 8sThe patch does not generate ASF License warnings.
30m 53s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 5b84c82466fd 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count79 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jun 23, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 27sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 10smaster passed
+1 💚compile0m 35smaster passed
+1 💚shadedjars4m 53sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 14sthe patch passed
+1 💚compile0m 36sthe patch passed
+1 💚javac0m 36sthe patch passed
+1 💚shadedjars4m 53spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22sthe patch passed
_ Other Tests _
-1 ❌unit218m 50shbase-server in the patch failed.
238m 54s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 8eb371fafdbb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/testReport/
Max. process+thread count4507 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jun 23, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 51sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 33smaster passed
+1 💚compile0m 47smaster passed
+1 💚shadedjars4m 35sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 36sthe patch passed
+1 💚compile0m 47sthe patch passed
+1 💚javac0m 47sthe patch passed
+1 💚shadedjars4m 36spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
+1 💚unit218m 49shbase-server in the patch passed.
240m 31s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5304
JIRA IssueHBASE-26630
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 097381b00799 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / da171c3
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/testReport/
Max. process+thread count4472 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5304/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@bbeaudreaultbbeaudreault changed the title HBASE-26630: Prevent early FS scan terminationHBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.nextRaw contractJun 23, 2023
@bbeaudreault
bbeaudreault merged commit 66fd6db into apache:masterJun 23, 2023
@bbeaudreault

bbeaudreault commented Jun 23, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

I updated this PR and the resulting merged commit message to link it to a new jira HBASE-27950. I realized that the old jira actually was not a problem, since the TableSnapshotInputFormat had been fixed to force ReadType.STREAM back in HBASE-26273, before HBASE-26630 was submitted. It was worth a new jira here so that we could still update ClientSideRegionScanner to honor the contract of RegionScanner.nextRaw for future proofing.

@hgromer
hgromer deleted the HBASE-26630 branch June 23, 2023 13:55
bbeaudreault pushed a commit that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HBASE-27950ClientSideRegionScanner does not adhere to RegionScanner.…
7d8e187
…nextRaw contract (#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
bbeaudreault pushed a commit that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HBASE-27950ClientSideRegionScanner does not adhere to RegionScanner.…
8b0dfc1
…nextRaw contract (#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
bbeaudreault pushed a commit that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HBASE-27950ClientSideRegionScanner does not adhere to RegionScanner.…
d839fd4
…nextRaw contract (#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
@Apache9

Apache9 commented Jun 23, 2023

Copy link
Copy Markdown
Contributor

I do not think force a stream read can fix the problem described in HBASE-26630?

The problem is that we should use the return value of nextRaw to determine whether there are still more values. This can not be fixed by changing the ReadType. And in HBASE-26273, we still allow users to change the ReadType back to PREAD, so it should not be considered as a 'fix' even if it can fix the problem right?

Opening a new issue is OK, but let's resolve HBASE-26630 as duplicated by HBASE-27950.

Thanks.

@bbeaudreault

bbeaudreault commented Jun 23, 2023

Copy link
Copy Markdown
Contributor

Yea, good point. Thanks, I'll fix the resolution.

bbeaudreault pushed a commit to HubSpot/hbase that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.…
704693d
…nextRaw contract (apache#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
bbeaudreault pushed a commit that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HBASE-27950ClientSideRegionScanner does not adhere to RegionScanner.…
5d58928
…nextRaw contract (#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
bbeaudreault pushed a commit to HubSpot/hbase that referenced this pull request Jun 23, 2023
@hgromer@bbeaudreault
HubSpot Backport: HBASE-27950 ClientSideRegionScanner does not adhere…
65a0a9f
… to RegionScanner.nextRaw contract (apache#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
@hgromerJenkins
HBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.…
c9e0697
…nextRaw contract (apache#5304)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit d839fd4)
Change-Id: I3d81cb0a667f4b572cbd4bac9265b4713e739f07
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bbeaudreaultbbeaudreaultbbeaudreault approved these changes
@Apache9Apache9Apache9 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hgromer@Apache-HBase@bbeaudreault@Apache9