Skip to content

[fix](mtmv) Fix partition track column fail when 'select *' used in related partition side - #43531

Merged
morrySnow merged 1 commit into
apache:masterfrom
seawinde:fix_partition_column_track_fail
Nov 14, 2024
Merged

[fix](mtmv) Fix partition track column fail when 'select *' used in related partition side#43531
morrySnow merged 1 commit into
apache:masterfrom
seawinde:fix_partition_column_track_fail

Conversation

@seawinde

@seawindeseawinde commented Nov 8, 2024

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #34781

Problem Summary:

Table def as following, if create partition mv as following will throw exception

ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a suitable base table for partitioning, the fail reason is can't not find valid partition track column, because

CREATE MATERIALIZED VIEW mv_10086
BUILD IMMEDIATE REFRESH AUTO ON MANUAL
partition by(l_orderkey)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES ('replication_num'='1') ASselect l1.*, O_CUSTKEY
from lineitem_list_partition l1
left outer join orders_list_partition
onl1.l_shipdate= o_orderdate;
CREATETABLE `orders_list_partition` (
`o_orderkey`BIGINTnot NULL,
`o_custkey`INTNULL,
`o_orderstatus`VARCHAR(1) NULL,
`o_totalprice`DECIMAL(15, 2) NULL,
`o_orderpriority`VARCHAR(15) NULL,
`o_clerk`VARCHAR(15) NULL,
`o_shippriority`INTNULL,
`o_comment`VARCHAR(79) NULL,
`o_orderdate`DATENULL
) ENGINE=OLAP
DUPLICATE KEY(`o_orderkey`, `o_custkey`)
COMMENT 'OLAP'
PARTITION BY list(o_orderkey) (
PARTITION p1 VALUESin ('1'),
PARTITION p2 VALUESin ('2'),
PARTITION p3 VALUESin ('3'),
PARTITION p4 VALUESin ('4')
)
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 3
PROPERTIES (
"replication_num"="1"
);
CREATETABLE `lineitem_list_partition` (
`l_orderkey`BIGINTnot NULL,
`l_linenumber`INTNULL,
`l_partkey`INTNULL,
`l_suppkey`INTNULL,
`l_quantity`DECIMAL(15, 2) NULL,
`l_extendedprice`DECIMAL(15, 2) NULL,
`l_discount`DECIMAL(15, 2) NULL,
`l_tax`DECIMAL(15, 2) NULL,
`l_returnflag`VARCHAR(1) NULL,
`l_linestatus`VARCHAR(1) NULL,
`l_commitdate`DATENULL,
`l_receiptdate`DATENULL,
`l_shipinstruct`VARCHAR(25) NULL,
`l_shipmode`VARCHAR(10) NULL,
`l_comment`VARCHAR(44) NULL,
`l_shipdate`DATENULL
) ENGINE=OLAP
DUPLICATE KEY(l_orderkey, l_linenumber, l_partkey, l_suppkey )
COMMENT 'OLAP'
PARTITION BY list(l_orderkey) (
PARTITION p1 VALUESin ('1'),
PARTITION p2 VALUESin ('2'),
PARTITION p3 VALUESin ('3')
)
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 3
PROPERTIES ("replication_num"="1"
);

Release note

Fix partition track column fail when 'select *' used in related partition side

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde

Copy link
Copy Markdown
MemberAuthor

run buildall

@wm1581066wm1581066 added the usercase Important user case type label label Nov 9, 2024
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2024
@morrySnow
morrySnow merged commit 89ef50a into apache:masterNov 14, 2024
github-actionsBot pushed a commit that referenced this pull request Nov 14, 2024
…elated partition side (#43531)
### What problem does this PR solve?
Related PR: #34781
Problem Summary:
Table def as following, if create partition mv as following will throw
exception
ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a
suitable base table for partitioning, the fail reason is can't not find
valid partition track column
CREATE MATERIALIZED VIEW mv_10086
BUILD IMMEDIATE REFRESH AUTO ON MANUAL
partition by(l_orderkey)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES ('replication_num' = '1') AS select l1.*, O_CUSTKEY
from lineitem_list_partition l1
left outer join orders_list_partition
on l1.l_shipdate = o_orderdate;
CREATE TABLE `orders_list_partition` (
`o_orderkey` BIGINT not NULL,
`o_custkey` INT NULL,
`o_orderstatus` VARCHAR(1) NULL,
`o_totalprice` DECIMAL(15, 2) NULL,
`o_orderpriority` VARCHAR(15) NULL,
`o_clerk` VARCHAR(15) NULL,
`o_shippriority` INT NULL,
`o_comment` VARCHAR(79) NULL,
`o_orderdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(`o_orderkey`, `o_custkey`)
COMMENT 'OLAP'
PARTITION BY list(o_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3'),
PARTITION p4 VALUES in ('4')
)
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 3
PROPERTIES (
"replication_num" = "1"
);
CREATE TABLE `lineitem_list_partition` (
`l_orderkey` BIGINT not NULL,
`l_linenumber` INT NULL,
`l_partkey` INT NULL,
`l_suppkey` INT NULL,
`l_quantity` DECIMAL(15, 2) NULL,
`l_extendedprice` DECIMAL(15, 2) NULL,
`l_discount` DECIMAL(15, 2) NULL,
`l_tax` DECIMAL(15, 2) NULL,
`l_returnflag` VARCHAR(1) NULL,
`l_linestatus` VARCHAR(1) NULL,
`l_commitdate` DATE NULL,
`l_receiptdate` DATE NULL,
`l_shipinstruct` VARCHAR(25) NULL,
`l_shipmode` VARCHAR(10) NULL,
`l_comment` VARCHAR(44) NULL,
`l_shipdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(l_orderkey, l_linenumber, l_partkey, l_suppkey )
COMMENT 'OLAP'
PARTITION BY list(l_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3')
)
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 3
PROPERTIES ("replication_num" = "1"
);
### Release note
Fix partition track column fail when 'select *' used in related
partition side
github-actionsBot pushed a commit that referenced this pull request Nov 14, 2024
…elated partition side (#43531)
### What problem does this PR solve?
Related PR: #34781
Problem Summary:
Table def as following, if create partition mv as following will throw
exception
ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a
suitable base table for partitioning, the fail reason is can't not find
valid partition track column
CREATE MATERIALIZED VIEW mv_10086
BUILD IMMEDIATE REFRESH AUTO ON MANUAL
partition by(l_orderkey)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES ('replication_num' = '1') AS select l1.*, O_CUSTKEY
from lineitem_list_partition l1
left outer join orders_list_partition
on l1.l_shipdate = o_orderdate;
CREATE TABLE `orders_list_partition` (
`o_orderkey` BIGINT not NULL,
`o_custkey` INT NULL,
`o_orderstatus` VARCHAR(1) NULL,
`o_totalprice` DECIMAL(15, 2) NULL,
`o_orderpriority` VARCHAR(15) NULL,
`o_clerk` VARCHAR(15) NULL,
`o_shippriority` INT NULL,
`o_comment` VARCHAR(79) NULL,
`o_orderdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(`o_orderkey`, `o_custkey`)
COMMENT 'OLAP'
PARTITION BY list(o_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3'),
PARTITION p4 VALUES in ('4')
)
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 3
PROPERTIES (
"replication_num" = "1"
);
CREATE TABLE `lineitem_list_partition` (
`l_orderkey` BIGINT not NULL,
`l_linenumber` INT NULL,
`l_partkey` INT NULL,
`l_suppkey` INT NULL,
`l_quantity` DECIMAL(15, 2) NULL,
`l_extendedprice` DECIMAL(15, 2) NULL,
`l_discount` DECIMAL(15, 2) NULL,
`l_tax` DECIMAL(15, 2) NULL,
`l_returnflag` VARCHAR(1) NULL,
`l_linestatus` VARCHAR(1) NULL,
`l_commitdate` DATE NULL,
`l_receiptdate` DATE NULL,
`l_shipinstruct` VARCHAR(25) NULL,
`l_shipmode` VARCHAR(10) NULL,
`l_comment` VARCHAR(44) NULL,
`l_shipdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(l_orderkey, l_linenumber, l_partkey, l_suppkey )
COMMENT 'OLAP'
PARTITION BY list(l_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3')
)
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 3
PROPERTIES ("replication_num" = "1"
);
### Release note
Fix partition track column fail when 'select *' used in related
partition side
dataroaring pushed a commit that referenced this pull request Nov 14, 2024
…*' used in related partition side #43531 (#43940)
Cherry-picked from #43531
Co-authored-by: seawinde <wusi@selectdb.com>
morrySnow pushed a commit that referenced this pull request Nov 19, 2024
…*' used in related partition side #43531 (#43941)
Cherry-picked from #43531
Co-authored-by: seawinde <wusi@selectdb.com>
hubgeter pushed a commit to hubgeter/doris that referenced this pull request Mar 12, 2025
…h wrongly when has filter (apache#3649)
pr: apache#43531
commitId: b2e42f2
pr: apache#43539 commitId: eb6c00cb0cecf4
---------
Co-authored-by: seawinde <wusi@selectdb.com>
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
…elated partition side (apache#43531)
### What problem does this PR solve?
Related PR: apache#34781
Problem Summary:
Table def as following, if create partition mv as following will throw
exception
ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a
suitable base table for partitioning, the fail reason is can't not find
valid partition track column
CREATE MATERIALIZED VIEW mv_10086
BUILD IMMEDIATE REFRESH AUTO ON MANUAL
partition by(l_orderkey)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES ('replication_num' = '1') AS select l1.*, O_CUSTKEY
from lineitem_list_partition l1
left outer join orders_list_partition
on l1.l_shipdate = o_orderdate;
CREATE TABLE `orders_list_partition` (
`o_orderkey` BIGINT not NULL,
`o_custkey` INT NULL,
`o_orderstatus` VARCHAR(1) NULL,
`o_totalprice` DECIMAL(15, 2) NULL,
`o_orderpriority` VARCHAR(15) NULL,
`o_clerk` VARCHAR(15) NULL,
`o_shippriority` INT NULL,
`o_comment` VARCHAR(79) NULL,
`o_orderdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(`o_orderkey`, `o_custkey`)
COMMENT 'OLAP'
PARTITION BY list(o_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3'),
PARTITION p4 VALUES in ('4')
)
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 3
PROPERTIES (
"replication_num" = "1"
);
CREATE TABLE `lineitem_list_partition` (
`l_orderkey` BIGINT not NULL,
`l_linenumber` INT NULL,
`l_partkey` INT NULL,
`l_suppkey` INT NULL,
`l_quantity` DECIMAL(15, 2) NULL,
`l_extendedprice` DECIMAL(15, 2) NULL,
`l_discount` DECIMAL(15, 2) NULL,
`l_tax` DECIMAL(15, 2) NULL,
`l_returnflag` VARCHAR(1) NULL,
`l_linestatus` VARCHAR(1) NULL,
`l_commitdate` DATE NULL,
`l_receiptdate` DATE NULL,
`l_shipinstruct` VARCHAR(25) NULL,
`l_shipmode` VARCHAR(10) NULL,
`l_comment` VARCHAR(44) NULL,
`l_shipdate` DATE NULL
) ENGINE=OLAP
DUPLICATE KEY(l_orderkey, l_linenumber, l_partkey, l_suppkey )
COMMENT 'OLAP'
PARTITION BY list(l_orderkey) (
PARTITION p1 VALUES in ('1'),
PARTITION p2 VALUES in ('2'),
PARTITION p3 VALUES in ('3')
)
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 3
PROPERTIES ("replication_num" = "1"
);
### Release note
Fix partition track column fail when 'select *' used in related
partition side
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.8-mergeddev/3.0.3-mergedreviewedusercaseImportant user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@seawinde@doris-robot@zddr@morrySnow@wm1581066@dataroaring