Skip to content

[ADDENDUM] HBASE-28458 BucketCache.notifyFileCachingCompleted may incorrectly consider a file fully cached (#5777) - #5791

Merged
wchevreuil merged 2 commits into
apache:masterfrom
wchevreuil:HBASE-28458-ADDENDUM
Apr 5, 2024
Merged

[ADDENDUM] HBASE-28458 BucketCache.notifyFileCachingCompleted may incorrectly consider a file fully cached (#5777)#5791
wchevreuil merged 2 commits into
apache:masterfrom
wchevreuil:HBASE-28458-ADDENDUM

Conversation

@wchevreuil

@wchevreuilwchevreuil commented Apr 2, 2024

Copy link
Copy Markdown
Contributor

While backporting original PR to branch-2, I still got some failures for TestBucketCachePersister.testPrefetchBlockEvictionWhilePrefetchRunning. I believe I was using the wrong offset in BucketCache.notifyFileCachingCompleted, so if the eviction happens while in the loop of notifyFileCachingCompleted, we fail to acknowledge the eviction and still consider the file fully cached.

@wchevreuil
[ADDENDUM] HBASE-28458 BucketCache.notifyFileCachingCompleted may inc…
ad123d3
…orrectly consider a file fully cached (apache#5777)
Change-Id: I681581b91000cf1f84253ad66f7195776793b364
@Apache-HBase

Apache-HBase commented Apr 2, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 44sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 58smaster passed
+1 💚compile3m 23smaster passed
+1 💚checkstyle0m 51smaster passed
+1 💚spotless0m 53sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 56smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 37sthe patch passed
+1 💚compile3m 16sthe patch passed
+1 💚javac3m 16sthe patch passed
+1 💚checkstyle0m 40sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck6m 28sPatch does not cause any errors with Hadoop 3.3.6.
+1 💚spotless0m 55spatch has no errors when running spotless:check.
+1 💚spotbugs2m 12sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
36m 19s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5791
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux cb08b37a27d0 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 90639d7
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count79 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 2, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 27sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 4smaster passed
+1 💚compile0m 52smaster passed
+1 💚shadedjars5m 29sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 50sthe patch passed
+1 💚compile0m 52sthe patch passed
+1 💚javac0m 52sthe patch passed
+1 💚shadedjars5m 28spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
-1 ❌unit207m 30shbase-server in the patch failed.
232m 7s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6f90fd44c565 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 90639d7
Default JavaEclipse Adoptium-17.0.10+7
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/testReport/
Max. process+thread count4480 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 47sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 10smaster passed
+1 💚compile0m 51smaster passed
+1 💚shadedjars6m 19sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 26sthe patch passed
+1 💚compile0m 56sthe patch passed
+1 💚javac0m 56sthe patch passed
+1 💚shadedjars6m 12spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
-1 ❌unit327m 57shbase-server in the patch failed.
355m 50s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 1ab86dd474b4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 90639d7
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/testReport/
Max. process+thread count4644 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 14sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 44smaster passed
+1 💚compile0m 57smaster passed
+1 💚shadedjars8m 13sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 50smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 14sthe patch passed
+1 💚compile1m 33sthe patch passed
+1 💚javac1m 33sthe patch passed
+1 💚shadedjars8m 33spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45sthe patch passed
_ Other Tests _
-1 ❌unit329m 24shbase-server in the patch failed.
364m 4s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 860d06905acc 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 90639d7
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/testReport/
Max. process+thread count4590 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@wchevreuil
anpother UT fix
d5ab023
Change-Id: Ifcd7fe5595cbf0d54c597c5ec72b504cc9ebbfae
@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 36sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 5smaster passed
+1 💚compile2m 28smaster passed
+1 💚checkstyle0m 37smaster passed
+1 💚spotless0m 45sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 33smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 49sthe patch passed
+1 💚compile2m 27sthe patch passed
+1 💚javac2m 27sthe patch passed
+1 💚checkstyle0m 32sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck4m 56sPatch does not cause any errors with Hadoop 3.3.6.
+1 💚spotless0m 42spatch has no errors when running spotless:check.
+1 💚spotbugs1m 37sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
28m 33s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5791
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux e7c2d6cbd788 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 28c1e3b
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count82 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 29sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 4smaster passed
+1 💚compile0m 50smaster passed
+1 💚shadedjars5m 24sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 45sthe patch passed
+1 💚compile0m 51sthe patch passed
+1 💚javac0m 51sthe patch passed
+1 💚shadedjars5m 24spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
+1 💚unit206m 18shbase-server in the patch passed.
230m 17s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 4211bf8d8283 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 28c1e3b
Default JavaEclipse Adoptium-17.0.10+7
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/testReport/
Max. process+thread count5206 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 14sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 1smaster passed
+1 💚compile0m 50smaster passed
+1 💚shadedjars5m 15sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile0m 51sthe patch passed
+1 💚javac0m 51sthe patch passed
+1 💚shadedjars5m 16spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
+1 💚unit227m 3shbase-server in the patch passed.
250m 50s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b900ccbbbd53 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 28c1e3b
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/testReport/
Max. process+thread count4718 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 3, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 37sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 41smaster passed
+1 💚compile0m 42smaster passed
+1 💚shadedjars5m 10sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 28sthe patch passed
+1 💚compile0m 40sthe patch passed
+1 💚javac0m 40sthe patch passed
+1 💚shadedjars5m 9spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
+1 💚unit235m 33shbase-server in the patch passed.
258m 11s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5791
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 406b3a25c01c 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 28c1e3b
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/testReport/
Max. process+thread count5396 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5791/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@wchevreuil
wchevreuil merged commit aea7e7c into apache:masterApr 5, 2024
wchevreuil added a commit that referenced this pull request Apr 5, 2024
@wchevreuil
[ADDENDUM]HBASE-28458BucketCache.notifyFileCachingCompleted may inc…
d7566ab
…orrectly consider a file fully cached (#5777) (#5791)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
wchevreuil added a commit that referenced this pull request Apr 5, 2024
@wchevreuil
[ADDENDUM]HBASE-28458BucketCache.notifyFileCachingCompleted may inc…
50eb7da
…orrectly consider a file fully cached (#5777) (#5791)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit d7566ab)
wchevreuil added a commit that referenced this pull request Apr 5, 2024
@wchevreuil
[ADDENDUM]HBASE-28458BucketCache.notifyFileCachingCompleted may inc…
a56126c
…orrectly consider a file fully cached (#5777) (#5791)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit d7566ab)
jhungund pushed a commit to janardhanrh/hbase that referenced this pull request May 29, 2024
@wchevreuil
CDPD-68550: Backport [ADDENDUM] HBASE-28458 BucketCache.notifyFileCac…
f28001c
…hingCompleted may incorrectly consider a file fully cached (apache#5777) (apache#5791)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit d7566ab)
Change-Id: I27599d2c82cf2dea074bba8caa1e6ba5b14b764a
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@petersomogyipetersomogyipetersomogyi approved these changes
@bbeaudreaultbbeaudreaultAwaiting requested review from bbeaudreault
@Apache9Apache9Awaiting requested review from Apache9

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wchevreuil@Apache-HBase@petersomogyi