Skip to content

PHOENIX-6400 Do no use local index with uncovered columns in the WHERE clause.#1158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lhofhansl merged 1 commit into from
Mar 4, 2021
Merged

PHOENIX-6400 Do no use local index with uncovered columns in the WHERE clause. #1158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 28 additions & 9 deletions phoenix-core/src/it/java/org/apache/phoenix/end2end/index/LocalIndexIT.java
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -80,6 +80,26 @@ public LocalIndexIT(boolean isNamespaceMapped) {
super(isNamespaceMapped);
}

@Test
public void testSelectFromIndexWithAdditionalWhereClause() throws Exception {
String tableName = schemaName + "." + generateUniqueName();
String indexName = "IDX_" + generateUniqueName();

Connection conn = getConnection();
conn.setAutoCommit(true);
if (isNamespaceMapped) {
conn.createStatement().execute("CREATE SCHEMA IF NOT EXISTS " + schemaName);
}

conn.createStatement().execute("CREATE TABLE " + tableName + " (pk INTEGER PRIMARY KEY, v1 FLOAT, v2 FLOAT)");
conn.createStatement().execute("CREATE LOCAL INDEX " + indexName + " ON " + tableName + "(v1)");
conn.createStatement().execute("UPSERT INTO " + tableName + " VALUES(1, 0.01, 1.0)");
ResultSet rs = conn.createStatement().executeQuery("SELECT COUNT(*) FROM "+tableName+" WHERE v1 < 0.1 and v2 < 10.0");
rs.next();
assertEquals(1, rs.getInt(1));
rs.close();
}

@Test
public void testDeleteFromLocalIndex() throws Exception {
String tableName = schemaName + "." + generateUniqueName();
Expand DownExpand Up@@ -212,13 +232,13 @@ public void testUseUncoveredLocalIndexWithPrefix() throws Exception {
QueryUtil.getExplainPlan(rs));
rs.close();

// 4. Longer prefix on the index, use it.
// 4. Longer prefix on the index.
// Note: This cannot use the local index, see PHOENIX-6300
rs = conn.createStatement().executeQuery("EXPLAIN SELECT v2 FROM " + tableName + " WHERE pk1 = 3 AND pk2 = 4 AND v1 = 3 AND v3 = 1");
assertEquals(
"CLIENT PARALLEL 1-WAY RANGE SCAN OVER "
+ physicalTableName + " [1,3,4,3]\n"
+ " SERVER FILTER BY FIRST KEY ONLY AND \"V3\" = 1\n"
+ "CLIENT MERGE SORT",
+ physicalTableName + " [3,4]\n"
+ " SERVER FILTER BY (V1 = 3.0 AND V3 = 1)",
QueryUtil.getExplainPlan(rs));
rs.close();
}
Expand DownExpand Up@@ -354,13 +374,12 @@ public void testUseUncoveredLocalIndex() throws Exception {
QueryUtil.getExplainPlan(rs));
rs.close();

// 10. Use index even when also filtering on non-indexed column
// 10. Cannot use index even when also filtering on non-indexed column, see PHOENIX-6400
rs = conn.createStatement().executeQuery("EXPLAIN SELECT * FROM " + tableName + " WHERE v2 = 2 AND v1 = 3");
assertEquals(
"CLIENT PARALLEL 1-WAY RANGE SCAN OVER "
+ indexPhysicalTableName + " [1,2]\n"
+ " SERVER FILTER BY FIRST KEY ONLY AND \"V1\" = 3.0\n"
+ "CLIENT MERGE SORT",
"CLIENT PARALLEL 1-WAY FULL SCAN OVER "
+ indexPhysicalTableName + "\n"
+ " SERVER FILTER BY (V2 = 2.0 AND V1 = 3.0)",
QueryUtil.getExplainPlan(rs));
rs.close();

Expand Down
11 changes: 11 additions & 0 deletions phoenix-core/src/main/java/org/apache/phoenix/compile/WhereCompiler.java
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@
import org.apache.phoenix.schema.PColumnFamily;
import org.apache.phoenix.schema.PTable;
import org.apache.phoenix.schema.PTable.ImmutableStorageScheme;
import org.apache.phoenix.schema.PTable.IndexType;
import org.apache.phoenix.schema.PTable.QualifierEncodingScheme;
import org.apache.phoenix.schema.PTable.ViewType;
import org.apache.phoenix.schema.PTableType;
Expand DownExpand Up@@ -200,6 +201,16 @@ protected ColumnRef resolveColumn(ColumnParseNode node) throws SQLException {
return ref;
}
PTable table = ref.getTable();
// If current table in the context is local index and table in column reference is global that
// means the column is not present in the local index. Local indexes do not currently support this.
// Throwing this exception here will cause this plan to be ignored when enumerating possible plans
// during the optimizing phase.
if (context.getCurrentTable().getTable().getIndexType() == IndexType.LOCAL
&& (table.getIndexType() == null || table.getIndexType() == IndexType.GLOBAL)) {
String schemaNameStr = table.getSchemaName()==null?null:table.getSchemaName().getString();
String tableNameStr = table.getTableName()==null?null:table.getTableName().getString();
throw new ColumnNotFoundException(schemaNameStr, tableNameStr, null, ref.getColumn().getName().getString());

@kadirozdekadirozdeMar 3, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not clear how returning ColumnNotFoundException results in skipping a plan. Can we add some comments here for that?

@lhofhansllhofhanslMar 3, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just brought that code back from before PHOENIX-5109.
It throws an exception, which is then caught in query enumeration in the optimizing phase and consequently ignores that plan.
Happy to add a comment to that extent.

@kadirozdekadirozdeMar 3, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like, "This exception will be caught in query enumeration in the optimizing phase and consequently the plan will be ignored" as you wrote, will be helpful. I will then approve it. Thanks!

@lhofhansllhofhanslMar 4, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this?

@kadirozdekadirozdeMar 4, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

}
// Track if we need to compare KeyValue during filter evaluation
// using column family. If the column qualifier is enough, we
// just use that.
Expand Down