Skip to content

HBASE-28900: Avoid resetting the bucket cache during recovery from persistence. - #6360

Merged
wchevreuil merged 1 commit into
apache:masterfrom
janardhanrh:HBASE-28900
Oct 14, 2024
Merged

HBASE-28900: Avoid resetting the bucket cache during recovery from persistence.#6360
wchevreuil merged 1 commit into
apache:masterfrom
janardhanrh:HBASE-28900

Conversation

@jhungund

Copy link
Copy Markdown
Contributor

When an inconsistency is detected during the recovery of bucket cache from persistence, we tend to throw away the complete cache and try to rebuild the cache. Thsi inconsistency can occur when the region server is abruptly terminated.

This can be avoided by skipping the inconsistent backing map entry and continuing with the cache recovery. The inconsistent backing map entry will be discarded during the subsequent bucket cache validation.

Change-Id: I41237bda2189cbd73e22e4eadf4d53d16a3733f6

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

"Couldn't find match for index " + idx + " in free list");
if (!matchFound) {
LOG.warn("We found more entries for bucket than the bucket capacity. "
+ "Skipping entry at cache offset {}", offset);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's add more info about the bucket. Something like:

LOG.warn("We found more entries for bucket starting at offset {} for blocks of {} size. "
+ "Skipping entry at cache offset {}", baseOffset, itemAllocationSize, offset);

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack!

…rsistence.
When an inconsistency is detected during the recovery of bucket cache from persistence,
we tend to throw away the complete cache and try to rebuild the cache.
Thsi inconsistency can occur when the region server is abruptly terminated.
This can be avoided by skipping the inconsistent backing map entry and continuing with
the cache recovery. The inconsistent backing map entry will be discarded during the subsequent
bucket cache validation.
Change-Id: I41237bda2189cbd73e22e4eadf4d53d16a3733f6
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec4m 3sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall2m 59smaster passed
+1 💚compile2m 58smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚spotbugs1m 33smaster passed
+1 💚spotless0m 44sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 55sthe patch passed
+1 💚compile2m 59sthe patch passed
+1 💚javac2m 59sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 38shbase-server: The patch generated 0 new + 25 unchanged - 1 fixed = 25 total (was 26)
+1 💚spotbugs1m 40sthe patch passed
+1 💚hadoopcheck10m 34sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
39m 9s
SubsystemReport/Notes
DockerClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6360/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6360
JIRA IssueHBASE-28900
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux e548135ed439 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c39455e
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6360/4/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 28sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 27smaster passed
+1 💚compile0m 55smaster passed
+1 💚javadoc0m 28smaster passed
+1 💚shadedjars5m 40sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 57sthe patch passed
+1 💚compile0m 55sthe patch passed
+1 💚javac0m 56sthe patch passed
-0 ⚠️javadoc0m 26s/results-javadoc-javadoc-hbase-server.txthbase-server generated 7 new + 48 unchanged - 0 fixed = 55 total (was 48)
+1 💚shadedjars5m 34spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit213m 45shbase-server in the patch passed.
238m 33s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6360/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6360
JIRA IssueHBASE-28900
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 49ef70866917 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c39455e
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6360/4/testReport/
Max. process+thread count5562 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6360/4/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@wchevreuil
wchevreuil merged commit cac58ba into apache:masterOct 14, 2024
wchevreuil pushed a commit that referenced this pull request Oct 14, 2024
…rsistence. (#6360)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
jhungund added a commit to janardhanrh/hbase that referenced this pull request Oct 14, 2024
…rsistence. (apache#6360)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
jhungund added a commit to janardhanrh/hbase that referenced this pull request Oct 15, 2024
…rsistence. (apache#6360)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Change-Id: If7e7a76723cf7d2d6e8c9e2ce0aeccfdba81c0bb
wchevreuil pushed a commit that referenced this pull request Oct 15, 2024
…rsistence. (#6360) (#6365)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
wchevreuil pushed a commit that referenced this pull request Oct 15, 2024
…rsistence. (#6360) (#6365)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jhungund@Apache-HBase@wchevreuil