Skip to content

PHOENIX-6323 isRawFilterSupported() returns true for HBase 2.1 comapt… - #1095

Closed
stoty wants to merge 1 commit into
apache:masterfrom
stoty:PHOENIX-6323
Closed

PHOENIX-6323 isRawFilterSupported() returns true for HBase 2.1 comapt…#1095
stoty wants to merge 1 commit into
apache:masterfrom
stoty:PHOENIX-6323

Conversation

@stoty

@stotystoty commented Jan 18, 2021

Copy link
Copy Markdown
Contributor

… module

@stoty
stoty requested a review from gjacoby126January 18, 2021 06:50
@stoty

stoty commented Jan 18, 2021

Copy link
Copy Markdown
ContributorAuthor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 19sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep4m 22sMaven dependency ordering for branch
+1 💚mvninstall12m 42smaster passed
+0hbaserecompile24m 5sHBase recompiled.
+1 💚compile1m 37smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚javadoc0m 35smaster passed
+0 🆗spotbugs0m 35sphoenix-hbase-compat-2.2.1 in master has 1 extant spotbugs warnings.
+0 🆗spotbugs0m 31sphoenix-hbase-compat-2.1.6 in master has 1 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall9m 43sthe patch passed
+0hbaserecompile16m 47sHBase recompiled.
+1 💚compile1m 36sthe patch passed
+1 💚javac1m 36sthe patch passed
-1 ❌checkstyle0m 39sroot: The patch generated 2 new + 15 unchanged - 3 fixed = 17 total (was 18)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 33sthe patch passed
+1 💚spotbugs1m 27sthe patch passed
_ Other Tests _
-1 ❌unit124m 28sroot in the patch failed.
+1 💚asflicense0m 55sThe patch does not generate ASF License warnings.
179m 7s
ReasonTests
Failed junit testsphoenix.end2end.ProductMetricsIT
phoenix.end2end.UpsertSelectIT
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1095
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
unameLinux 0ee514011613 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revisionmaster / 50df995
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/1/artifact/yetus-general-check/output/patch-unit-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/1/testReport/
Max. process+thread count9525 (vs. ulimit of 30000)
modulesC: phoenix-hbase-compat-2.2.1 phoenix-hbase-compat-2.1.6 U: .
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@gjacoby126

gjacoby126 commented Jan 20, 2021

Copy link
Copy Markdown
Contributor

@stoty - it would be very good if we could support the raw filter scans in the HBase 2.2 branch. They're required to fix a serious bug where building an index on a view can't apply the view's WHERE clause filter condition properly to the index, and so you wind up indexing the entire table in each view index, wasting lots of storage by creating tons of corrupt index rows (that are never read). @abhishek-chouhan discovered and fixed it in PHOENIX-5932.

Does requiring 2.2.5 on the 2.2 branch create difficulties?

@stoty

stoty commented Jan 20, 2021•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Thanks for the explanation, @gjacoby126.

For reference, these are our options:

  • bump the minimum version to 2.2.5

  • split the 2.2 pofile into two

  • or make isRawFilterSupported() dynamic depending on the detected HBase version

  • bumping the version is the simplest, but we lock out some users from Phoenix.

  • splitting the profile adds a bit of bloat, and possible confusion

  • making isRawFilterSupported() dynamic is easy, but will cause divergent behaviour for different HBase versions.

I'm going to update the PR to bump the version requirement, but LMK if you prefer one of the other versions.

@stoty

stoty commented Jan 20, 2021

Copy link
Copy Markdown
ContributorAuthor

New version requires HBase 2.2.5

@stoty

stoty commented Jan 20, 2021

Copy link
Copy Markdown
ContributorAuthor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 2 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep4m 37sMaven dependency ordering for branch
+1 💚mvninstall13m 12smaster passed
+0hbaserecompile26m 3sHBase recompiled.
+1 💚compile1m 43smaster passed
+1 💚checkstyle0m 53smaster passed
+1 💚javadoc3m 9smaster passed
+0 🆗spotbugs0m 35sphoenix-hbase-compat-2.2.1 in master has 1 extant spotbugs warnings.
+0 🆗spotbugs0m 35sphoenix-hbase-compat-2.1.6 in master has 1 extant spotbugs warnings.
+0 🆗spotbugs5m 1sroot in master has 1004 extant spotbugs warnings.
+0 🆗spotbugs3m 27sphoenix-core in master has 955 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 33sMaven dependency ordering for patch
+1 💚mvninstall11m 39sthe patch passed
+0hbaserecompile20m 16sHBase recompiled.
+1 💚compile1m 39sthe patch passed
+1 💚javac1m 39sthe patch passed
+1 💚checkstyle0m 42sroot: The patch generated 0 new + 215 unchanged - 2 fixed = 215 total (was 217)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚xml0m 3sThe patch has no ill-formed XML file.
+1 💚javadoc2m 52sthe patch passed
+1 💚spotbugs10m 42sthe patch passed
_ Other Tests _
-1 ❌unit192m 54sroot in the patch failed.
-1 ❌asflicense2m 35sThe patch generated 610 ASF License warnings.
280m 28s
ReasonTests
Failed junit testsphoenix.end2end.index.GlobalMutableNonTxIndexWithLazyPostBatchWriteIT
phoenix.end2end.index.txn.RollbackIT
phoenix.tx.ParameterizedTransactionIT
phoenix.tx.FlappingTransactionIT
phoenix.end2end.QueryIT
phoenix.end2end.salted.SaltedTableUpsertSelectIT
phoenix.replication.SystemCatalogWALEntryFilterIT
phoenix.end2end.index.ImmutableIndexExtendedIT
phoenix.end2end.ViewTTLIT
phoenix.rpc.UpdateCacheConnectionLevelPropIT
phoenix.util.IndexScrutinyIT
phoenix.end2end.UpsertSelectIT
phoenix.iterate.PhoenixQueryTimeoutIT
phoenix.end2end.join.HashJoinLocalIndexIT
phoenix.rpc.UpdateCacheIT
phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
phoenix.end2end.index.IndexCoprocIT
phoenix.end2end.join.WhereOptimizerForJoinFiltersIT
phoenix.end2end.index.MutableIndexSplitForwardScanIT
phoenix.iterate.RoundRobinResultIteratorIT
phoenix.end2end.salted.SaltedTableIT
phoenix.end2end.join.SortMergeJoinNoIndexIT
phoenix.end2end.index.SingleCellIndexIT
phoenix.end2end.join.HashJoinPersistentCacheIT
phoenix.end2end.index.LocalMutableNonTxIndexIT
phoenix.end2end.join.SubqueryIT
phoenix.end2end.index.SaltedIndexIT
phoenix.end2end.index.MutableIndexIT
phoenix.end2end.RowValueConstructorOffsetIT
phoenix.end2end.DisableLocalIndexIT
phoenix.end2end.ToCharFunctionIT
phoenix.end2end.join.SortMergeJoinLocalIndexIT
phoenix.end2end.join.HashJoinGlobalIndexIT
phoenix.tx.TxCheckpointIT
phoenix.end2end.join.HashJoinNoIndexIT
phoenix.iterate.ChunkedResultIteratorIT
phoenix.end2end.index.InvalidIndexStateClientSideIT
phoenix.end2end.join.SortMergeJoinGlobalIndexIT
phoenix.tx.TransactionIT
phoenix.end2end.QueryExecWithoutSCNIT
phoenix.end2end.join.SubqueryUsingSortMergeJoinIT
phoenix.end2end.index.GlobalMutableTxIndexIT
phoenix.end2end.PrimitiveTypeIT
phoenix.end2end.index.MutableIndexSplitReverseScanIT
phoenix.end2end.join.HashJoinMoreIT
phoenix.monitoring.IndexMetricsIT
phoenix.end2end.index.txn.MutableRollbackIT
phoenix.end2end.UpsertWithSCNIT
phoenix.trace.PhoenixTableMetricsWriterIT
phoenix.end2end.OnDuplicateKeyIT
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1095
Optional Testsdupname asflicense javac javadoc unit xml compile spotbugs hbaserebuild hbaseanti checkstyle
unameLinux 8cd070844151 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revisionmaster / 772ff2c
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/2/artifact/yetus-general-check/output/patch-unit-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/2/testReport/
asflicensehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/2/artifact/yetus-general-check/output/patch-asflicense-problems.txt
Max. process+thread count8336 (vs. ulimit of 30000)
modulesC: phoenix-hbase-compat-2.2.5 phoenix-hbase-compat-2.1.6 phoenix-core . U: .
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1095/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stotystoty closed this Jan 21, 2021
@stoty
stoty deleted the PHOENIX-6323 branch July 29, 2021 14:09
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@abhishek-chouhanabhishek-chouhanabhishek-chouhan approved these changes
@gjacoby126gjacoby126Awaiting requested review from gjacoby126

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@stoty@gjacoby126@abhishek-chouhan