Skip to content

branch-4.0: [Feature] doris cross-cluster query #57898 - #58571

Merged
yiguolei merged 2 commits into
branch-4.0from
auto-pick-57898-branch-4.0
Dec 9, 2025
Merged

branch-4.0: [Feature] doris cross-cluster query #57898#58571
yiguolei merged 2 commits into
branch-4.0from
auto-pick-57898-branch-4.0

Conversation

@github-actions

@github-actionsgithub-actionsBot commented Dec 1, 2025

Copy link
Copy Markdown
Contributor

Cherry-picked from #57898

@hello-stephen

hello-stephen commented Dec 1, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaringdataroaring reopened this Dec 1, 2025
@hello-stephen

hello-stephen commented Dec 1, 2025

Copy link
Copy Markdown
Contributor

run buildall

morningman
morningman previously approved these changes Dec 1, 2025
View reviewed changes
@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 1, 2025
@github-actions

github-actionsBot commented Dec 1, 2025

Copy link
Copy Markdown
ContributorAuthor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Dec 1, 2025

Copy link
Copy Markdown
ContributorAuthor

PR approved by anyone and no changes requested.

@hello-stephen

hello-stephen commented Dec 1, 2025

Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage83.49% (1573/1884)
Line Coverage67.69% (28063/41459)
Region Coverage68.17% (13814/20264)
Branch Coverage58.44% (7369/12610)

@hello-stephen

hello-stephen commented Dec 1, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.70% (18212/34560)
Line Coverage38.09% (165825/435345)
Region Coverage33.09% (128676/388854)
Branch Coverage33.93% (55462/163461)

@HonestManXin@morningman
[Feature] doris cross-cluster query (#57898)
4d547dc
Related issue: #57884
```
MySQL [demo]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | LiveSince |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
| fe_a7c0b6d8_82c2_48f0_8220_fb65dd18be69 | 10.37.75.124 | 9010 | 8030 | 9030 | 9020 | 8070 | FOLLOWER | true | 742250121 | true | true | 2409 | 2025-11-11 14:42:16 | 2025-11-11 14:44:06 | true | | doris-0.0.0-009c3b552a | Yes | 2025-11-11 14:42:16 |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
1 row in set (0.016 sec)
MySQL [demo]> show catalog edoris;
+-----------------+-------------------------------+
| Key | Value |
+-----------------+-------------------------------+
| create_time | 2025-11-11 11:25:33.488106853 |
| fe_arrow_hosts | 10.37.103.28:8070 |
| fe_http_hosts | 10.37.103.28:8030 |
| fe_thrift_hosts | 10.37.103.28:9020 |
| password | *XXX |
| type | doris |
| use_meta_cache | true |
| user | test |
+-----------------+-------------------------------+
8 rows in set (0.002 sec)
MySQL [demo]> select * from inner_table;
+----------+--------+
| log_type | reason |
+----------+--------+
| 2 | inner2 |
| 3 | inner3 |
| 4 | inner4 |
+----------+--------+
3 rows in set (0.032 sec)
MySQL [demo]> select * from edoris.external.example_tbl_duplicate;
+---------------------+----------+------------+-----------+-------+---------------------+
| log_time | log_type | error_code | error_msg | op_id | op_time |
+---------------------+----------+------------+-----------+-------+---------------------+
| 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 |
+---------------------+----------+------------+-----------+-------+---------------------+
1 row in set (0.059 sec)
MySQL [demo]> select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type);
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
| log_type | reason | log_time | log_type | error_code | error_msg | op_id | op_time |
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
| 2 | inner2 | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 |
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
1 row in set (0.050 sec)
MySQL [demo]> explain select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type) where error_code=2;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| log_type[#16] |
| reason[#17] |
| log_time[#18] |
| log_type[#19] |
| error_code[#20] |
| error_msg[#21] |
| op_id[#22] |
| op_time[#23] |
| PARTITION: HASH_PARTITIONED: log_type[#6] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 3:VHASH JOIN(200) |
| | join op: INNER JOIN(BROADCAST)[] |
| | equal join conjunct: (log_type[#6] = log_type[#1]) |
| | cardinality=3 |
| | vec output tuple id: 3 |
| | output tuple id: 3 |
| | vIntermediate tuple ids: 2 |
| | hash output slot ids: 0 1 2 3 4 5 6 7 |
| | runtime filters: RF000[min_max] <- log_type[#1](1/1/1048576), RF001[in_or_bloom] <- log_type[#1](1/1/1048576) |
| | final projections: log_type[#8], reason[#9], log_time[#10], log_type[#11], error_code[#12], error_msg[#13], op_id[#14], op_time[#15] |
| | final project output tuple id: 3 |
| | distribute expr lists: log_type[#6] |
| | distribute expr lists: |
| | |
| |----1:VEXCHANGE |
| | offset: 0 |
| | distribute expr lists: log_type[#1] |
| | |
| 2:VOlapScanNode(187) |
| TABLE: demo.inner_table(inner_table), PREAGGREGATION: ON |
| partitions=1/1 (inner_table) |
| tablets=1/1, tabletList=1762832514491 |
| cardinality=3, avgRowSize=901.6666, numNodes=1 |
| pushAggOp=NONE |
| runtime filters: RF000[min_max] -> log_type[#6], RF001[in_or_bloom] -> log_type[#6] |
| |
| PLAN FRAGMENT 1 |
| |
| PARTITION: HASH_PARTITIONED: log_type[#1] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 01 |
| UNPARTITIONED |
| |
| 0:VOlapScanNode(188) |
| TABLE: external.example_tbl_duplicate(example_tbl_duplicate), PREAGGREGATION: ON |
| PREDICATES: (error_code[#2] = 2) |
| partitions=1/1 (example_tbl_duplicate) |
| tablets=1/1, tabletList=1762481736238 |
| cardinality=1, avgRowSize=7425.0, numNodes=1 |
| pushAggOp=NONE |
| |
| |
| |
| ========== STATISTICS ========== |
| planed with unknown column statistics |
+-------------------------------------------------------------------------------------------------------------------------------------------+
65 rows in set (0.040 sec)
```
@morningman
morningmanforce-pushed the auto-pick-57898-branch-4.0 branch from 5d239d0 to 4d547dcCompareDecember 2, 2025 02:50
@morningman

morningman commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

hello-stephen commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage83.49% (1573/1884)
Line Coverage67.64% (28043/41459)
Region Coverage68.13% (13805/20264)
Branch Coverage58.37% (7361/12610)

@morningman

morningman commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

run buildall

@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Dec 2, 2025
@doris-robot

doris-robot commented Dec 2, 2025

Copy link
Copy Markdown

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage83.49% (1573/1884)
Line Coverage67.64% (28042/41459)
Region Coverage68.17% (13814/20264)
Branch Coverage58.41% (7365/12610)

@doris-robot

doris-robot commented Dec 2, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.70% (18216/34568)
Line Coverage38.07% (165749/435400)
Region Coverage33.09% (128719/388966)
Branch Coverage33.91% (55444/163487)

@yiguolei
yiguolei merged commit 5cd3734 into branch-4.0Dec 9, 2025
24 of 30 checks passed
@github-actions
github-actionsBot deleted the auto-pick-57898-branch-4.0 branch December 9, 2025 09:19
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes
@morningmanmorningmanmorningman left review comments

Assignees

No one assigned

Labels

reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen@morningman@doris-robot@yiguolei@dataroaring@HonestManXin