Skip to content

HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1 - #251

Closed
apurtell wants to merge 1 commit into
apache:branch-1from
apurtell:HBASE-22114-branch-1
Closed

HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1#251
apurtell wants to merge 1 commit into
apache:branch-1from
apurtell:HBASE-22114-branch-1

Conversation

@apurtell

@apurtellapurtell commented May 23, 2019

Copy link
Copy Markdown
Contributor

HBASE-15560 W-TinyLFU based BlockCache (Ben Manes)

@apurtell
apurtell requested review from busbey and lhofhanslMay 23, 2019 18:39
@apurtell

apurtell commented May 23, 2019

Copy link
Copy Markdown
ContributorAuthor

Just opening a PR for this backport. I think we are blocked on issues with precommit handling of the JDK version specific conditional build stuff that could permanently break precommit, but not any issue with this patch itself.

@Apache-HBase

This comment has been minimized.

Sign in to view
busbey
busbey requested changes May 24, 2019
View reviewed changes

@busbeybusbey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can make time this holiday weekend to work out the qabot things.

will do a more thorough review as a part of that as well.

Comment threadhbase-tinylfu-blockcache/pom.xml Outdated
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from c928fc6 to ffc4f3eCompareJune 2, 2019 18:16
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from ffc4f3e to a8a5b16CompareJune 3, 2019 18:59
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from a8a5b16 to f4a176dCompareJune 10, 2019 19:31
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from f4a176d to 29613fcCompareJune 12, 2019 19:57
@Apache-HBase

This comment has been minimized.

Sign in to view
busbey
busbey approved these changes Jun 13, 2019
View reviewed changes

@busbeybusbey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat, pushing my updates worked.

the changes to hbase-personality will need to go to all branches. this should fix most of the complaints from qa bot. it won't fix the xml thing (it's a long standing issue on jdk7 branches), and I haven't dug into what's up with the unit tests that failed.

Apache-HBase
Apache-HBase reviewed Jun 13, 2019
View reviewed changes
Comment threaddev-support/hbase-personality.sh Outdated
# on branches-1, the hbase-tinylfu-blockcache only works for jdk8. so if it changes
# run whatever test is requested at the top level so that maven can handle only including
# the module when the correct jdk is around.
if [[ "${MODULES[*]}" =~ hbase-tinylfu-blockcache ]]; then

@Apache-HBaseApache-HBaseJun 13, 2019

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shellcheck:25: warning: Did you forget to close this double quoted string? [SC1078]

@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from e27842f to a57a936CompareJune 18, 2019 17:04
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from a57a936 to fb4513bCompareJune 26, 2019 01:30
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from fb4513b to 9abf0b8CompareJuly 11, 2019 01:32
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from 9abf0b8 to b7d6af8CompareJuly 13, 2019 01:26
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache9

Apache9 commented Jul 25, 2019

Copy link
Copy Markdown
Contributor

Let's finish this? @apurtell

@busbey

busbey commented Jul 25, 2019

Copy link
Copy Markdown
Contributor

Was this waiting on me and handling the jdk7/jdk8 bit? If so, sorry been swamped. I can block out some time this week to give it another go.

@saintstack

saintstack commented Sep 10, 2020

Copy link
Copy Markdown
Contributor

@apurtell close?

@Apache-HBase

This comment has been minimized.

Sign in to view
saintstack
saintstack reviewed Oct 6, 2020
View reviewed changes

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we close out this PR since it has been hanging out a while @apurtell ? Thanks.

@saintstack

saintstack commented Oct 6, 2020

Copy link
Copy Markdown
Contributor

Oh, had a push 13 days ago so not so old....

@Apache-HBase

This comment has been minimized.

Sign in to view
@busbey

busbey commented Oct 16, 2020

Copy link
Copy Markdown
Contributor

I would really like to not close this out. Iirc it's just waiting on an update to our CI tooling to properly handle jdk dependent modules. That's debt that has a hard time finding help, to be sure.

@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from e5c0921 to a263f0bCompareNovember 3, 2020 20:15
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from 4daa600 to 06b3055CompareDecember 4, 2020 19:12
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from 06b3055 to b70c26bCompareApril 8, 2021 17:51
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from b70c26b to 2028e56CompareMay 2, 2021 21:32
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from 2028e56 to aa17212CompareJune 20, 2021 17:40
@Apache-HBase

This comment has been minimized.

Sign in to view
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from aa17212 to 4be8fd0CompareJune 22, 2021 01:04
@Apache-HBase

This comment has been minimized.

Sign in to view
@busbeybusbey added the backport This PR is a back port of some issue or issues already committed to master label Jun 24, 2021
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from 4be8fd0 to c1a597bCompareNovember 3, 2021 19:06
@Apache-HBase

Apache-HBase commented Nov 3, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 58sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
-1 ❌hbaseanti0m 0sThe patch appears use Hadoop classification instead of HBase.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗mvndep2m 54sMaven dependency ordering for branch
+1 💚mvninstall8m 12sbranch-1 passed
+1 💚compile2m 11sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile1m 57sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle7m 42sbranch-1 passed
+0 🆗refguide3m 55sbranch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars3m 13sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc3m 37sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc4m 16sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs2m 50sUsed deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗findbugs0m 27sbranch/hbase-resource-bundle no findbugs output file (findbugsXml.xml)
_ Patch Compile Tests _
+0 🆗mvndep0m 26sMaven dependency ordering for patch
+1 💚mvninstall2m 9sthe patch passed
-1 ❌compile0m 10sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
-1 ❌javac0m 10sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚compile2m 0sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac2m 0sthe patch passed
+1 💚checkstyle4m 59sroot: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
-1 ❌xml0m 1sThe patch has 5 ill-formed XML file(s).
+0 🆗refguide2m 59spatch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars3m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck5m 12sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
-1 ❌javadoc0m 10sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚javadoc2m 44sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗findbugs0m 14shbase-resource-bundle has no data from findbugs
-1 ❌findbugs0m 10shbase-tinylfu-blockcache in the patch failed.
_ Other Tests _
-1 ❌unit147m 55sroot in the patch failed.
+1 💚asflicense0m 39sThe patch does not generate ASF License warnings.
249m 43s
ReasonTests
XMLParsing Error(s):
hbase-common/src/main/resources/hbase-default.xml
hbase-it/pom.xml
hbase-resource-bundle/src/main/resources/supplemental-models.xml
hbase-tinylfu-blockcache/pom.xml
pom.xml
Failed junit testshadoop.hbase.replication.TestReplicationSource
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/Dockerfile
GITHUB PR#251
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml
unameLinux 0ff5ffaccee3 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh
git revisionbranch-1 / 40b4cb1
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
refguidehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/branch-site/book.html
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
xmlhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/xml.txt
refguidehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-site/book.html
javadochttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-javadoc-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
findbugshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-findbugs-hbase-tinylfu-blockcache.txt
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-unit-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/testReport/
Max. process+thread count4453 (vs. ulimit of 10000)
modulesC: hbase-common hbase-resource-bundle hbase-server hbase-it . hbase-tinylfu-blockcache U: .
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@busbey

busbey commented Nov 4, 2021

Copy link
Copy Markdown
Contributor

FYI stack I'll be taking a look at getting this PR merged over the weekend.

Andrew Purtell@apurtell
HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
0a0ff9f
HBASE-15560 W-TinyLFU based BlockCache (Ben Manes)
@apurtell
apurtellforce-pushed the HBASE-22114-branch-1 branch from c1a597b to 0a0ff9fCompareDecember 7, 2021 00:05
@Apache-HBase

Apache-HBase commented Dec 7, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 3sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
-1 ❌hbaseanti0m 0sThe patch appears use Hadoop classification instead of HBase.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗mvndep2m 55sMaven dependency ordering for branch
+1 💚mvninstall8m 30sbranch-1 passed
+1 💚compile2m 14sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile2m 0sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle7m 48sbranch-1 passed
+0 🆗refguide4m 11sbranch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars4m 19sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc4m 26sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc5m 20sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs3m 41sUsed deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗findbugs0m 29sbranch/hbase-resource-bundle no findbugs output file (findbugsXml.xml)
_ Patch Compile Tests _
+0 🆗mvndep0m 32sMaven dependency ordering for patch
+1 💚mvninstall2m 49sthe patch passed
-1 ❌compile0m 11sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
-1 ❌javac0m 11sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚compile2m 48sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac2m 48sthe patch passed
+1 💚checkstyle6m 54sroot: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
-1 ❌xml0m 1sThe patch has 5 ill-formed XML file(s).
+0 🆗refguide4m 11spatch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars4m 8spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck7m 1sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
-1 ❌javadoc0m 10sroot in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚javadoc3m 26sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗findbugs0m 16shbase-resource-bundle has no data from findbugs
-1 ❌findbugs0m 10shbase-tinylfu-blockcache in the patch failed.
_ Other Tests _
-1 ❌unit156m 35sroot in the patch failed.
+1 💚asflicense0m 39sThe patch does not generate ASF License warnings.
281m 29s
ReasonTests
XMLParsing Error(s):
hbase-common/src/main/resources/hbase-default.xml
hbase-it/pom.xml
hbase-resource-bundle/src/main/resources/supplemental-models.xml
hbase-tinylfu-blockcache/pom.xml
pom.xml
Failed junit testshadoop.hbase.replication.TestReplicationSource
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/Dockerfile
GITHUB PR#251
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml
unameLinux 1cda19d72a10 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh
git revisionbranch-1 / 8d96fc3
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
refguidehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/branch-site/book.html
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
xmlhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/xml.txt
refguidehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-site/book.html
javadochttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-javadoc-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
findbugshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-findbugs-hbase-tinylfu-blockcache.txt
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-unit-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/testReport/
Max. process+thread count5029 (vs. ulimit of 10000)
modulesC: hbase-common hbase-resource-bundle hbase-server hbase-it . hbase-tinylfu-blockcache U: .
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9

Apache9 commented Jan 17, 2022

Copy link
Copy Markdown
Contributor

Any progress here?

@Apache9Apache9 closed this Aug 9, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@busbeybusbeybusbey approved these changes
@saintstacksaintstacksaintstack left review comments
@virajjasanivirajjasanivirajjasani approved these changes
@lhofhansllhofhanslAwaiting requested review from lhofhansl
+1 more reviewer
@Apache-HBaseApache-HBaseApache-HBase left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backportThis PR is a back port of some issue or issues already committed to master

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@apurtell@Apache-HBase@Apache9@busbey@ndimiduk@saintstack@Reidddddd@virajjasani