Skip to content

HBASE-27646 Should not use pread when prefetching in HFilePreadReader - #5063

Merged
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-27646
Mar 20, 2023
Merged

HBASE-27646 Should not use pread when prefetching in HFilePreadReader#5063
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-27646

Conversation

@sunhelly

Copy link
Copy Markdown
Contributor

Using isolate stream reader for prefetching in pread reader, to speed up the prefetching progress.
The blocks cached when prefetching will be released after the pread reader is closed.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 27smaster passed
+1 💚compile2m 52smaster passed
+1 💚checkstyle0m 43smaster passed
+1 💚spotless0m 50sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 54smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile2m 48sthe patch passed
+1 💚javac2m 48sthe patch passed
+1 💚checkstyle0m 40sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck14m 21sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs1m 48sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
45m 7s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5063
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 66c5a43ab9ac 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 10037df
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 50sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 16smaster passed
+1 💚compile0m 56smaster passed
+1 💚shadedjars6m 30sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 32smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 14sthe patch passed
+1 💚compile0m 53sthe patch passed
+1 💚javac0m 53sthe patch passed
+1 💚shadedjars6m 7spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21sthe patch passed
_ Other Tests _
-1 ❌unit234m 19shbase-server in the patch failed.
263m 16s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux dcfbb7d6b779 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 10037df
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/testReport/
Max. process+thread count2663 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 38sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall5m 52smaster passed
+1 💚compile1m 14smaster passed
+1 💚shadedjars6m 39sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 24sthe patch passed
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
+1 💚shadedjars5m 43spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28sthe patch passed
_ Other Tests _
-1 ❌unit262m 2shbase-server in the patch failed.
293m 58s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux faddcf3016a5 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 10037df
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/testReport/
Max. process+thread count2500 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 35smaster passed
+1 💚compile2m 24smaster passed
+1 💚checkstyle0m 35smaster passed
+1 💚spotless0m 44sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 22sthe patch passed
+1 💚compile2m 20sthe patch passed
+1 💚javac2m 20sthe patch passed
+1 💚checkstyle0m 34sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 43sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 40spatch has no errors when running spotless:check.
+1 💚spotbugs1m 34sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
38m 16s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5063
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux ed095061148e 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 12sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 42smaster passed
+1 💚compile0m 42smaster passed
+1 💚shadedjars4m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 12sthe patch passed
+1 💚compile0m 41sthe patch passed
+1 💚javac0m 41sthe patch passed
+1 💚shadedjars4m 32spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22sthe patch passed
_ Other Tests _
+1 💚unit199m 35shbase-server in the patch passed.
224m 0s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5273a7ac57e5 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/testReport/
Max. process+thread count2601 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 14sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 9smaster passed
+1 💚compile0m 38smaster passed
+1 💚shadedjars5m 9sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 58sthe patch passed
+1 💚compile0m 42sthe patch passed
+1 💚javac0m 42sthe patch passed
+1 💚shadedjars5m 30spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21sthe patch passed
_ Other Tests _
+1 💚unit209m 14shbase-server in the patch passed.
234m 15s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 924caba0124e 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/testReport/
Max. process+thread count2641 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/io/FileLink.java Outdated
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 15sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall5m 32smaster passed
+1 💚compile3m 29smaster passed
+1 💚checkstyle0m 48smaster passed
+1 💚spotless1m 7sbranch has no errors when running spotless:check.
+1 💚spotbugs2m 22smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 17sthe patch passed
+1 💚compile3m 26sthe patch passed
+1 💚javac3m 26sthe patch passed
+1 💚checkstyle1m 5sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck18m 33sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 59spatch has no errors when running spotless:check.
+1 💚spotbugs2m 21sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
56m 39s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5063
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux c669005bedaf 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37858bb
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 25smaster passed
+1 💚compile0m 39smaster passed
+1 💚shadedjars4m 36sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 45sthe patch passed
+1 💚compile0m 35sthe patch passed
+1 💚javac0m 35sthe patch passed
+1 💚shadedjars4m 34spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21sthe patch passed
_ Other Tests _
+1 💚unit231m 22shbase-server in the patch passed.
253m 26s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 65320223f263 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37858bb
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/testReport/
Max. process+thread count2649 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 58sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 34smaster passed
+1 💚compile0m 48smaster passed
+1 💚shadedjars4m 27sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 21sthe patch passed
+1 💚compile0m 46sthe patch passed
+1 💚javac0m 46sthe patch passed
+1 💚shadedjars4m 25spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
-1 ❌unit232m 6shbase-server in the patch failed.
255m 35s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5063
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 13dbdd67f000 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37858bb
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/testReport/
Max. process+thread count2684 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@sunhelly

Copy link
Copy Markdown
ContributorAuthor

The failed UTs are not relevant to this PR, and all passed in local tests.
Ping @bbeaudreault@Apache9 , if there are any other concerns, please let me know. Thanks.

@bbeaudreaultbbeaudreault left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks

@sunhelly
sunhelly merged commit 201846b into apache:masterMar 20, 2023
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5122)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5123)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5124)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
wesiq pushed a commit to web-iq/hbase that referenced this pull request Mar 27, 2023
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
…apache#5063) (apache#5123)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit fde0f4b)
Change-Id: Id727198b9ef22a293f0984521bd6816266b106cb
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@sunhelly@Apache-HBase@bbeaudreault@Apache9