Skip to content

HBASE-28195 set start row as prefix if a scan with PrefixFilter - #5514

Open
frostruan wants to merge 1 commit into
apache:masterfrom
frostruan:HBASE-28195
Open

HBASE-28195 set start row as prefix if a scan with PrefixFilter#5514
frostruan wants to merge 1 commit into
apache:masterfrom
frostruan:HBASE-28195

Conversation

@frostruan

Copy link
Copy Markdown
Contributor

This PR introduces a ScanRangeOptimizer to try to reduce unnecessary reading of data based on filters user set.

For example, if user want to scan data where rowkey > 'hhh' and rowkey < 'mmm', the optimizer can optimize start row to 'hhh' and stop row to 'mmm'. Compare to the default start row and stop row, EMPTY_START_ROW and EMPTY_STOP_ROW, this will help speed up scan request.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 23sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for branch
+1 💚mvninstall2m 30smaster passed
+1 💚compile0m 31smaster passed
+1 💚shadedjars5m 14sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall2m 18sthe patch passed
+1 💚compile0m 29sthe patch passed
+1 💚javac0m 29sthe patch passed
+1 💚shadedjars5m 10spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 24sthe patch passed
_ Other Tests _
+1 💚unit1m 49shbase-common in the patch passed.
-1 ❌unit1m 3shbase-client in the patch failed.
24m 3s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5514
Optional Testsjavac javadoc unit shadedjars compile
unameLinux dd7ad45a760d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e806350
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-client.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/testReport/
Max. process+thread count360 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 34sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for branch
+1 💚mvninstall3m 54smaster passed
+1 💚compile0m 47smaster passed
+1 💚shadedjars6m 15sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall3m 19sthe patch passed
+1 💚compile0m 40sthe patch passed
+1 💚javac0m 40sthe patch passed
+1 💚shadedjars6m 0spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 40sthe patch passed
_ Other Tests _
+1 💚unit2m 50shbase-common in the patch passed.
-1 ❌unit1m 27shbase-client in the patch failed.
28m 53s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5514
Optional Testsjavac javadoc unit shadedjars compile
unameLinux cb796ef6263c 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e806350
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-client.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/testReport/
Max. process+thread count370 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 18sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for branch
+1 💚mvninstall2m 51smaster passed
+1 💚compile1m 13smaster passed
+1 💚checkstyle0m 32smaster passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 16smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall2m 39sthe patch passed
+1 💚compile1m 12sthe patch passed
+1 💚javac1m 12sthe patch passed
-0 ⚠️checkstyle0m 16shbase-client: The patch generated 3 new + 4 unchanged - 0 fixed = 7 total (was 4)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 16sPatch does not cause any errors with Hadoop 3.2.4 3.3.6.
-1 ❌spotless0m 18spatch has 66 errors when running spotless:check, run spotless:apply to fix.
+1 💚spotbugs1m 31sthe patch passed
_ Other Tests _
+1 💚asflicense0m 19sThe patch does not generate ASF License warnings.
29m 46s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5514
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux c61adbdc9a94 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e806350
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5514/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9

Copy link
Copy Markdown
Contributor

There is a setStartStopRowForPrefixScan method for Scan already? I think it is exactly for the same purpose...

@frostruan

frostruan commented Nov 13, 2023

Copy link
Copy Markdown
ContributorAuthor

Thanks for reviewing Duo. Yes, the setStartStopRowForPrefixScan method works for prefix filtering, but it can not work for range filtering. Maybe the title misunderstood you. What I want to introduce here, is like the query optimizer sub-system in RDBMS. It will optimize the scan range based on the filters that user sets. For example, if user want to scan data where rowkey > 'hhh' and rowkey < 'mmm', the optimizer can optimize start row to 'hhh' and stop row to 'mmm'. Compare to the default start row and stop row, EMPTY_START_ROW and EMPTY_STOP_ROW, this will help speed up scan request.

@Apache9

Copy link
Copy Markdown
Contributor

Then let's change the title and post a simple design doc to discuss first? I think introducing a new mechanism is fine, but we need to discuss it first. At least, changing the Scan object passed in may break our users code...

@frostruan

Copy link
Copy Markdown
ContributorAuthor

OK. Thanks for your advise Duo. Let me prepare the design doc first.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@frostruan@Apache-HBase@Apache9