Skip to content

HBASE-25899 Improve efficiency of SnapshotHFileCleaner - #3280

Merged
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-25899-1
May 24, 2021
Merged

HBASE-25899 Improve efficiency of SnapshotHFileCleaner#3280
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-25899-1

Conversation

@sunhelly

@sunhellysunhelly commented May 19, 2021

Copy link
Copy Markdown
Contributor

More details in HBASE-25899

@Apache-HBase

Apache-HBase commented May 19, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 14smaster passed
+1 💚compile3m 23smaster passed
+1 💚checkstyle1m 11smaster passed
+1 💚spotbugs2m 11smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 5sthe patch passed
+1 💚compile3m 19sthe patch passed
+1 💚javac3m 19sthe patch passed
-0 ⚠️checkstyle1m 13shbase-server: The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck20m 1sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 23sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
51m 41s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3280
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 4ddf24d14ee3 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5b99409
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 33sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 58smaster passed
+1 💚compile1m 19smaster passed
+1 💚shadedjars9m 8sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 44smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 49sthe patch passed
+1 💚compile1m 16sthe patch passed
+1 💚javac1m 16sthe patch passed
+1 💚shadedjars9m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41sthe patch passed
_ Other Tests _
-1 ❌unit208m 13shbase-server in the patch failed.
244m 41s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux f397ce057c26 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5b99409
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/testReport/
Max. process+thread count2649 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 1sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 19smaster passed
+1 💚compile1m 3smaster passed
+1 💚shadedjars8m 54sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 2sthe patch passed
+1 💚compile1m 2sthe patch passed
+1 💚javac1m 2sthe patch passed
+1 💚shadedjars8m 56spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
+1 💚unit216m 52shbase-server in the patch passed.
249m 14s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9c722476d6fb 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5b99409
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/testReport/
Max. process+thread count2812 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 20, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 7sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 12smaster passed
+1 💚compile3m 22smaster passed
+1 💚checkstyle1m 8smaster passed
+1 💚spotbugs2m 9smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 1sthe patch passed
+1 💚compile3m 20sthe patch passed
+1 💚javac3m 20sthe patch passed
-0 ⚠️checkstyle1m 8shbase-server: The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 54sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 23sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
51m 15s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3280
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 68e7bbd870e0 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / fe47557
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 20, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 9sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 55smaster passed
+1 💚compile1m 13smaster passed
+1 💚shadedjars8m 24sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 23sthe patch passed
+1 💚compile1m 15sthe patch passed
+1 💚javac1m 15sthe patch passed
+1 💚shadedjars8m 13spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42sthe patch passed
_ Other Tests _
-1 ❌unit141m 2shbase-server in the patch failed.
177m 14s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 4824a43a347d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / fe47557
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/testReport/
Max. process+thread count4035 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 20, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 32sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 56smaster passed
+1 💚compile1m 16smaster passed
+1 💚shadedjars10m 33sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 53sthe patch passed
+1 💚compile1m 17sthe patch passed
+1 💚javac1m 17sthe patch passed
+1 💚shadedjars10m 53spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43sthe patch passed
_ Other Tests _
-1 ❌unit157m 53shbase-server in the patch failed.
196m 11s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 1577ae02fbe6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / fe47557
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/testReport/
Max. process+thread count3624 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9
Apache9 requested changes May 20, 2021
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java Outdated
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java Outdated
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java
@Apache-HBase

Apache-HBase commented May 21, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 8sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 17smaster passed
+1 💚compile3m 24smaster passed
+1 💚checkstyle1m 10smaster passed
+1 💚spotbugs2m 15smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 2sthe patch passed
+1 💚compile3m 24sthe patch passed
+1 💚javac3m 24sthe patch passed
-0 ⚠️checkstyle1m 9shbase-server: The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck20m 15sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 45sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
52m 33s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3280
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 509089b13db5 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 15e8611
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 21, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 4sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 5smaster passed
+1 💚compile1m 1smaster passed
+1 💚shadedjars8m 55sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 2sthe patch passed
+1 💚compile1m 2sthe patch passed
+1 💚javac1m 2sthe patch passed
+1 💚shadedjars9m 0spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
+1 💚unit220m 38shbase-server in the patch passed.
252m 57s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 1b523a83c285 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 15e8611
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/testReport/
Max. process+thread count2681 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 21, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 10sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall6m 9smaster passed
+1 💚compile1m 34smaster passed
+1 💚shadedjars9m 55sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 48smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 45sthe patch passed
+1 💚compile1m 33sthe patch passed
+1 💚javac1m 33sthe patch passed
+1 💚shadedjars10m 15spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 50sthe patch passed
_ Other Tests _
+1 💚unit223m 56shbase-server in the patch passed.
265m 22s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3280
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 177c2b22aa85 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 15e8611
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/testReport/
Max. process+thread count2579 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3280/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@sunhelly
sunhelly merged commit b02c810 into apache:masterMay 24, 2021
asfgit pushed a commit that referenced this pull request May 24, 2021
@sunhelly
HBASE-25899Improve efficiency of SnapshotHFileCleaner (#3280)
76d56bc
Signed-off-by: Duo Zhang <zhangduo@apache.org>
taklwu
taklwu reviewed Dec 2, 2022
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java
Comment on lines +207 to 212
synchronized (this) {
refreshCache();
currentCache = cache;
refreshed = true;
}
}

@taklwutaklwuDec 2, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this line is still blocking the delete tasks... basically I LOG.info/print the lock information before the line of if (lock == null || lock.tryLock()) { , I found it won't add any unReferencedFiles file because once a synchronized refresh cache in line 207 is being called e.g. java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock@568e37c0[Locked by thread hfile_cleaner-dir-scan-pool-2] , then whatever files passing in will be skipped.

maybe I'm wondered, why do we need a write lock for refreshing snapshot file cache ?

vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
@sunhellyStephen Wu
CDPD-47258 Backport HBASE-25899 Improve efficiency of SnapshotHFileCl…
3221ce5
…eaner (apache#3280)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 76d56bc)
Change-Id: I74defca8babde5a941985d8e560aa1c58a5f5bd3
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@taklwutaklwutaklwu left review comments
@Apache9Apache9Apache9 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sunhelly@Apache-HBase@taklwu@Apache9