Skip to content

HBASE-26797 HBase 1.x clients will choke on rep_barrier rows when scanning hbase 2.x meta - #4161

Merged
apurtell merged 2 commits into
apache:branch-1from
HubSpot:HBASE-26797
Mar 4, 2022
Merged

HBASE-26797 HBase 1.x clients will choke on rep_barrier rows when scanning hbase 2.x meta#4161
apurtell merged 2 commits into
apache:branch-1from
HubSpot:HBASE-26797

Conversation

@bbeaudreault

Copy link
Copy Markdown
Contributor

I found it really hard to add a test case for this. I think i'd need to modify meta to add a dummy extra CF and then put a bad row in there and validate that it gets ignored by these reads. That should be straightforward, but alters to meta are disallowed in a few places. I could probably add some sort of internal-only config to disable those checks for the purpose of test, or maybe a setter which sets a volatile value? It's hard to access them from the TestingUtility, but maybe doable. Let me know if you'd like me to go through those efforts to test this, but I think it might not be worth it?

throws IOException {
byte[] searchRow = HRegionInfo.createRegionName(userTableName, row, HConstants.NINES, false);
Scan scan = Scan.createGetClosestRowOrBeforeReverseScan(searchRow);
scan.addFamily(HConstants.CATALOG_FAMILY);

@bbeaudreaultbbeaudreaultMar 3, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real error comes from ConnectionManager, but I tried to audit all reads against meta table and this was the only other one (that I could find) which did not have a family specified. I figured I should add it while I'm here even if we haven't hit an issue with this yet.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 39sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 1sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-0 ⚠️test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚mvninstall10m 9sbranch-1 passed
+1 💚compile0m 12sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile0m 15sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle0m 29sbranch-1 passed
+1 💚shadedjars2m 6sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 15sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs1m 13sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 12sbranch-1 passed
_ Patch Compile Tests _
+1 💚mvninstall1m 14sthe patch passed
+1 💚compile0m 11sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 11sthe patch passed
+1 💚compile0m 15sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 15sthe patch passed
+1 💚checkstyle0m 19sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars1m 49spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck3m 40sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 12sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 15sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚findbugs0m 55sthe patch passed
_ Other Tests _
+1 💚unit2m 3shbase-client in the patch passed.
+1 💚asflicense0m 17sThe patch does not generate ASF License warnings.
33m 29s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/1/artifact/out/Dockerfile
GITHUB PR#4161
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux f89617504b3b 5.4.0-1054-aws #57~18.04.1-Ubuntu SMP Thu Jul 15 03:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4161/out/precommit/personality/provided.sh
git revisionbranch-1 / 7bb40b3
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/1/testReport/
Max. process+thread count128 (vs. ulimit of 10000)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/1/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtellapurtell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but please add comments adjacent to the added statements with at least a reference to the JIRA

@bbeaudreault

Copy link
Copy Markdown
ContributorAuthor

Thanks for the review, I'll add that tomorrow.

@bbeaudreault

Copy link
Copy Markdown
ContributorAuthor

@apurtell done -- I'm assuming that's for added protection against a future regression, given the lack of a test. But correct me if I'm wrong, for my own edification.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 32sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗mvndep2m 48sMaven dependency ordering for branch
+1 💚mvninstall7m 51sbranch-1 passed
+1 💚compile0m 33sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile0m 42sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle1m 19sbranch-1 passed
+1 💚shadedjars2m 3sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 38sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs1m 37sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs2m 47sbranch-1 passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall1m 18sthe patch passed
+1 💚compile0m 34sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 34sthe patch passed
+1 💚compile0m 43sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 43sthe patch passed
+1 💚checkstyle1m 11sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
-1 ❌shadedjars1m 15spatch has 10 errors when building our shaded downstream artifacts.
+1 💚hadoopcheck3m 14sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 29sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 39sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚findbugs2m 41sthe patch passed
_ Other Tests _
+1 💚unit2m 5shbase-client in the patch passed.
-1 ❌unit115m 58shbase-server in the patch failed.
-1 ❌asflicense0m 37sThe patch generated 1 ASF License warnings.
157m 34s
ReasonTests
Failed junit testshadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL
hadoop.hbase.regionserver.TestRegionProcessRowsWithLocks
hadoop.hbase.replication.TestReplicationEndpoint
hadoop.hbase.client.TestLookupRegionMetaFamilyFilter
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/artifact/out/Dockerfile
GITHUB PR#4161
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 7b2cca033300 5.4.0-1054-aws #57~18.04.1-Ubuntu SMP Thu Jul 15 03:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4161/out/precommit/personality/provided.sh
git revisionbranch-1 / 7bb40b3
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
shadedjarshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/artifact/out/patch-shadedjars.txt
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/testReport/
asflicensehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/artifact/out/patch-asflicense-problems.txt
Max. process+thread count4078 (vs. ulimit of 10000)
modulesC: hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/2/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 0sDocker mode activated.
-1 ❌docker0m 12sDocker failed to build run-specific yetus/hbase:tp-8361}.
SubsystemReport/Notes
GITHUB PR#4161
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4161/3/console
versionsgit=2.17.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtell

Copy link
Copy Markdown
Contributor

Correct, its so we know not to remove them without thought toward this issue since there isn't a test for it

@apurtell
apurtell merged commit acc7299 into apache:branch-1Mar 4, 2022
@charlesconnell
charlesconnell deleted the HBASE-26797 branch September 19, 2025 15:18
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@bbeaudreault@Apache-HBase@apurtell