Skip to content

PHOENIX-6424 SELECT cf1.* FAILS with a WHERE clause including cf2. - #1177

Merged
lhofhansl merged 1 commit into
apache:masterfrom
lhofhansl:6424
Mar 23, 2021
Merged

PHOENIX-6424 SELECT cf1.* FAILS with a WHERE clause including cf2.#1177
lhofhansl merged 1 commit into
apache:masterfrom
lhofhansl:6424

Conversation

@lhofhansl

Copy link
Copy Markdown
Contributor

See Jira. In the SELECT CF1.* WHERE CF2.C ... cannot use the EncodedQualifiersColumnProjectionFilter since the number of qualifiers is not know (so the trackcolumns Bitset cannot be build ahead of time).

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 16sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall15m 40smaster passed
+0hbaserecompile22m 9sHBase recompiled.
+1 💚compile1m 0smaster passed
+1 💚checkstyle0m 49smaster passed
+1 💚javadoc0m 52smaster passed
+0 🆗spotbugs3m 3sphoenix-core in master has 958 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall8m 13sthe patch passed
+0hbaserecompile17m 40sHBase recompiled.
+1 💚compile1m 2sthe patch passed
+1 💚javac1m 2sthe patch passed
-1 ❌checkstyle0m 50sphoenix-core: The patch generated 15 new + 720 unchanged - 14 fixed = 735 total (was 734)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 50sthe patch passed
+1 💚spotbugs3m 15sthe patch passed
_ Other Tests _
-1 ❌unit106m 45sphoenix-core in the patch failed.
+1 💚asflicense0m 44sThe patch does not generate ASF License warnings.
163m 23s
ReasonTests
Failed junit testsphoenix.end2end.PermissionNSEnabledWithCustomAccessControllerIT
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1177
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
unameLinux d2443e3856be 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revisionmaster / b5bea88
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/1/testReport/
Max. process+thread count12917 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, left one minor comment if you would like to include multiple cols

} else {
// cannot use EncodedQualifiersColumnProjectionFilter in this case
// since there's an unknown set of qualifiers (cf.*)
trackedColumnsBitset = null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think this makes sense, and hopefully should work for multiple filters too. Shall we add one more test to cover multiple cols and filters? Something like
SELECT w.*, x.* FROM tableName WHERE y.v2='xyz' and z.v3='abc' ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... Lemme add that test and then merge.
Thanks @virajjasani

@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

I'll also check the test failure.

@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

Actually... When I add local indexes into the mix it still fails for another reason. I'll look into that as well. If it's an easy fix I'll add it here otherwise I'll file another jira.

@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

Added more tests as @virajjasani suggested. Also added local indexes to the test, and included an additional fix for local indexes.

* Disabling this optimization for tables with more than one column family.
* See PHOENIX-3890.
*/
return !scan.isRaw() && table.getColumnFamilies().size() <= 1 && table.getImmutableStorageScheme() != null

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This inadvertently caught local indexes too (which have 0 column families declared in their table definition)

@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

Expanded the test slightly to also do a CF.* from a column family with an indexed column.
This should be good now.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 31sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall19m 14smaster passed
+0hbaserecompile27m 18sHBase recompiled.
+1 💚compile1m 2smaster passed
+1 💚checkstyle0m 53smaster passed
+1 💚javadoc0m 51smaster passed
+0 🆗spotbugs3m 6sphoenix-core in master has 956 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall8m 8sthe patch passed
+0hbaserecompile17m 35sHBase recompiled.
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
-1 ❌checkstyle0m 53sphoenix-core: The patch generated 16 new + 833 unchanged - 15 fixed = 849 total (was 848)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 52sthe patch passed
+1 💚spotbugs3m 12sthe patch passed
_ Other Tests _
-1 ❌unit107m 52sphoenix-core in the patch failed.
+1 💚asflicense0m 49sThe patch does not generate ASF License warnings.
171m 9s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1177
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
unameLinux baa7eaee37b1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revisionmaster / 84d91ff
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/2/testReport/
Max. process+thread count13878 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

Last test run still looks good.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall15m 38smaster passed
+0hbaserecompile22m 4sHBase recompiled.
+1 💚compile1m 3smaster passed
+1 💚checkstyle0m 53smaster passed
+1 💚javadoc0m 54smaster passed
+0 🆗spotbugs3m 3sphoenix-core in master has 956 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall8m 13sthe patch passed
+0hbaserecompile17m 31sHBase recompiled.
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
-1 ❌checkstyle0m 53sphoenix-core: The patch generated 16 new + 833 unchanged - 15 fixed = 849 total (was 848)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 51sthe patch passed
+1 💚spotbugs3m 14sthe patch passed
_ Other Tests _
-1 ❌unit105m 28sphoenix-core in the patch failed.
+1 💚asflicense0m 44sThe patch does not generate ASF License warnings.
158m 12s
ReasonTests
Failed junit testsphoenix.end2end.PermissionNSEnabledIT
phoenix.end2end.PermissionNSEnabledWithCustomAccessControllerIT
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1177
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
unameLinux 90e46aeea902 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revisionmaster / 84d91ff
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/3/testReport/
Max. process+thread count13966 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1177/3/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lhofhansl
lhofhansl merged commit 2a2d996 into apache:masterMar 23, 2021
@lhofhansl

Copy link
Copy Markdown
ContributorAuthor

Thanks @virajjasani

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@lhofhansl@stoty@virajjasani