Skip to content

HBASE-28934: Avoid waits within HFilePreadReader constructors for cache initialisation. - #6389

Merged
wchevreuil merged 1 commit into
apache:masterfrom
janardhanrh:HBASE-28934
Oct 25, 2024
Merged

HBASE-28934: Avoid waits within HFilePreadReader constructors for cache initialisation.#6389
wchevreuil merged 1 commit into
apache:masterfrom
janardhanrh:HBASE-28934

Conversation

@jhungund

Copy link
Copy Markdown
Contributor

The current implementation is such that the HFilePreadReader constructor waits for the cache initialisation to decide whether or not, to create a prefetcher for an HFile.

However, as the HFilePreadReader constructor waits for the bucket cache initialisation, it blocks the region from bein opened. Any calls to read from the region fail until the cache is initialised.

This prevents the region from being usable, which otherwise, can read from the data store (cloud storage) to serve the client requests.

This change prevents the waits with the HFilePreadReader constructor and instead, move this wait to the prefetch thread, which waits until the cache is initialised to decide whether or, not to prefetch the file. Any requests to the regions will be served from the main data store instead of cache.

Also, the missed cache access during the cache initialisation should be accounted for the cache misses.

Change-Id: I952c592a1020874ad874f1909f776d8652375a1b

…he initialisation.
The current implementation is such that the HFilePreadReader constructor waits for
the cache initialisation to decide whether or not, to create a prefetcher for an HFile.
However, as the HFilePreadReader constructor waits for the bucket cache initialisation,
it blocks the region from bein opened. Any calls to read from the region fail until the
cache is initialised.
This prevents the region from being usable, which otherwise, can read from the data store
(cloud storage) to serve the client requests.
This change prevents the waits with the HFilePreadReader constructor and instead, move this
wait to the prefetch thread, which waits until the cache is initialised to decide whether or,
not to prefetch the file. Any requests to the regions will be served from the main data store
instead of cache.
Also, the missed cache access during the cache initialisation should be accounted for the
cache misses.
Change-Id: I952c592a1020874ad874f1909f776d8652375a1b
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 40sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall2m 57smaster passed
+1 💚compile2m 54smaster passed
+1 💚checkstyle0m 37smaster passed
+1 💚spotbugs1m 33smaster passed
+1 💚spotless0m 44sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 48sthe patch passed
+1 💚compile2m 58sthe patch passed
+1 💚javac2m 58sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 36sthe patch passed
+1 💚spotbugs1m 40sthe patch passed
+1 💚hadoopcheck10m 18sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
35m 2s
SubsystemReport/Notes
DockerClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6389/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6389
JIRA IssueHBASE-28934
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 630e4fe1adb5 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / a2a1d88
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6389/1/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 44sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 1smaster passed
+1 💚compile0m 56smaster passed
+1 💚javadoc0m 30smaster passed
+1 💚shadedjars5m 10sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile0m 56sthe patch passed
+1 💚javac0m 56sthe patch passed
+1 💚javadoc0m 29sthe patch passed
+1 💚shadedjars5m 6spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit246m 24s/patch-unit-hbase-server.txthbase-server in the patch failed.
270m 59s
SubsystemReport/Notes
DockerClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6389/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6389
JIRA IssueHBASE-28934
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 4fdbfe84e1c3 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / a2a1d88
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6389/1/testReport/
Max. process+thread count4601 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6389/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@wchevreuil

Copy link
Copy Markdown
Contributor

UT failure looks unrelated.

@wchevreuil
wchevreuil merged commit 6b5cc5b into apache:masterOct 25, 2024
wchevreuil pushed a commit that referenced this pull request Oct 25, 2024
…he initialisation. (#6389)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request Oct 25, 2024
…he initialisation. (apache#6389)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
wchevreuil added a commit that referenced this pull request Oct 28, 2024
…he initialisation. (#6389) (#6398)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Co-authored-by: jhungund <106576553+jhungund@users.noreply.github.com>
stoty pushed a commit to stoty/hbase that referenced this pull request Nov 22, 2025
… constructors for cache initialisation. (apache#6389) (apache#6398) (apache#589)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Co-authored-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Co-authored-by: jhungund <106576553+jhungund@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jhungund@Apache-HBase@wchevreuil