Support materialized view load and insert by HangyuanLiu · Pull Request #3677 · apache/doris

HangyuanLiu · 2020-05-25T02:02:43Z

This commit mainly supports load bitmap_union, hll_union, and count materialized views.

The main changes are as follows:
1、insert stmt support load extend column
2、load stmt support load extend column

Issue : #3344

…to mv_load

… mv_load

EmmyMiao87 · 2020-06-08T09:52:15Z

 }
+ // The extension column of the materialized view is added to the expression evaluation of load
+ for (Column column : tbl.getFullSchema()) {
+ if (column.getDefineExpr() != null) {


Are you sure that this is correctcolumn(a, tmp_c) set(b=f(c), c=b)？

Are you sure that this is correctcolumn(a, tmp_c) set(b=f(c), c=b)？
Fix done

EmmyMiao87 · 2020-06-08T11:41:43Z

Is it possible to unify the columns of schema change and materialized view into one logical process in insert stmt?

morningman

LGTM

EmmyMiao87 · 2020-06-15T03:51:15Z

+ * we should add them to the end of targetColumns.
+ * eg: origin targetColumns: (A,B,C), shadow column: __doris_materialized_view_bitmap_union_C
+ * after processing, targetColumns: (A, B, C, __doris_materialized_view_bitmap_union_C), and
+ * origColIdx2MVColumn has 1 element: "3, __doris_materialized_view_bitmap_union_C"


Suggested change
* origColIdx2MVColumnhas1element: "3, __doris_materialized_view_bitmap_union_C"
* origColIdx2MVColumnhas1element: "2, __doris_materialized_view_bitmap_union_C"

EmmyMiao87 · 2020-06-15T03:52:42Z

- String origName = ((SlotRef) slots.get(0)).getColumnName();
+SlotRef refColumn = column.getRefColumn();
+if (refColumn == null) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_BAD_FIELD_ERROR);


Better to show which field is error.

EmmyMiao87 · 2020-06-15T04:47:02Z

- queryStmt.getResultExprs().add(Expr.substituteList(Lists.newArrayList(expr), smap, analyzer, false).get(0));
+ Expr e = Expr.substituteList(Lists.newArrayList(mvColumn.getDefineExpr()), smap, analyzer, false).get(0);
+ queryStmt.getResultExprs().add(e);
+ queryStmt.getBaseTblResultExprs().add(e);


What's different between resultExpr and baseTblResultExpr

… mv_load

EmmyMiao87

LGTM

…to mv_load

support materialized view load
7846337

HangyuanLiu self-assigned this May 25, 2020

HangyuanLiu added area/materialized-view Issues or PRs related to materialized view kind/feature Categorizes issue or PR as related to a new feature. labels May 25, 2020

HangyuanLiuand others added 8 commits May 28, 2020 14:16

Merge branch 'master' ofhttps://github.com/apache/incubator-dorisin…
a94db58
…to mv_load

Merge branch 'master' ofhttps://github.com/apache/incubator-dorisin…
94bd0a0
…to mv_load

support materialized view insert statement
731164f

support materialized view insert statement
65e709c

Merge branch 'master' ofhttps://github.com/apache/incubator-dorisin…
208adb6
…to mv_load

support materialized view insert statement
5a6ae7c

Merge branch 'mv_load' of github.com:HangyuanLiu/incubator-doris into…
0ca77be
… mv_load

fix bug
2052dc4

imay reviewed Jun 5, 2020
View reviewed changes

Comment threadfe/src/main/java/org/apache/doris/analysis/InsertStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/analysis/InsertStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/load/Load.java Outdated

fix typo and comment
ae6fc56

HangyuanLiu changed the title ~~[WIP] Support materialized view load and insert~~Support materialized view load and insertJun 6, 2020

EmmyMiao87 requested changes Jun 8, 2020
View reviewed changes

EmmyMiao87 reviewed Jun 8, 2020
View reviewed changes

Comment threadfe/src/main/java/org/apache/doris/analysis/InsertStmt.java Outdated

HangyuanLiu force-pushed the mv_load branch from 31b8d96 to 6186e6eCompare June 9, 2020 07:59

fix bug
22d18bc

HangyuanLiu force-pushed the mv_load branch from 6186e6e to 22d18bcCompare June 10, 2020 13:26

morningman previously approved these changes Jun 13, 2020
View reviewed changes

EmmyMiao87 requested changes Jun 15, 2020
View reviewed changes

HangyuanLiu added 2 commits June 15, 2020 14:54

Merge branch 'mv_load' of github.com:HangyuanLiu/incubator-doris into…
49e89cc
… mv_load

unify shadow column and mv column
49d5900

HangyuanLiu dismissed morningman’s stale review via 49d5900June 15, 2020 07:15

fix bug : define expr should add cast node
4b6b589

EmmyMiao87 reviewed Jul 16, 2020
View reviewed changes

fix typo
29b8062

Merge branch 'mv_load' of github.com:HangyuanLiu/incubator-doris into…
fd6435b
… mv_load

EmmyMiao87 approved these changes Jul 16, 2020
View reviewed changes

EmmyMiao87 added the approved Indicates a PR has been approved by one committer. label Jul 16, 2020

Merge branch 'master' ofhttps://github.com/apache/incubator-dorisin…
a299b8b
…to mv_load

EmmyMiao87 merged commit de3c4b1 into apache:masterJul 17, 2020

EmmyMiao87 linked an issue Jul 17, 2020 that may be closed by this pull request
[Proposal] Materialized View 2.0 #3344
Closed

EmmyMiao87 mentioned this pull request Sep 1, 2020
Release Notes 0.13.0 #4370
Closed

	* origColIdx2MVColumnhas1element: "3, __doris_materialized_view_bitmap_union_C"
	* origColIdx2MVColumnhas1element: "2, __doris_materialized_view_bitmap_union_C"

Uh oh!

Conversation

HangyuanLiu commented May 25, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EmmyMiao87Jun 8, 2020

Choose a reason for hiding this comment

Uh oh!

HangyuanLiuJun 11, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EmmyMiao87 commented Jun 8, 2020

Uh oh!

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

EmmyMiao87Jun 15, 2020

Choose a reason for hiding this comment

Uh oh!

EmmyMiao87Jun 15, 2020

Choose a reason for hiding this comment

Uh oh!

EmmyMiao87Jun 15, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EmmyMiao87 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HangyuanLiu commented May 25, 2020•
edited
Loading