Uh oh!
There was an error while loading. Please reload this page.
[feature](nereids) Support dereference expression - #57532
Merged
Conversation
Thearas
commented
Oct 30, 2025
Contributor
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
924060929
commented
Oct 30, 2025
ContributorAuthor
run buildall |
1 similar comment
924060929
commented
Oct 30, 2025
ContributorAuthor
run buildall |
doris-robot
commented
Oct 30, 2025
ClickBench: Total hot run time: 29.12 s |
hello-stephen
commented
Oct 31, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
924060929
commented
Oct 31, 2025
ContributorAuthor
run buildall |
924060929
commented
Oct 31, 2025
ContributorAuthor
run buildall |
doris-robot
commented
Oct 31, 2025
ClickBench: Total hot run time: 27.68 s |
hello-stephen
commented
Oct 31, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
2 similar comments
hello-stephen
commented
Oct 31, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
hello-stephen
commented
Nov 1, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
924060929
commented
Nov 28, 2025
ContributorAuthor
run buildall |
doris-robot
commented
Nov 28, 2025
TPC-H: Total hot run time: 34057 ms |
doris-robot
commented
Nov 28, 2025
TPC-DS: Total hot run time: 181500 ms |
doris-robot
commented
Nov 28, 2025
ClickBench: Total hot run time: 27.33 s |
hello-stephen
commented
Nov 28, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
morrySnow
approved these changes
Nov 28, 2025
Contributor
PR approved by at least one committer and no changes requested. |
starocean999
approved these changes
Nov 28, 2025
Contributor
PR approved by anyone and no changes requested. |
hello-stephen
commented
Nov 28, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
Uh oh!
There was an error while loading. Please reload this page.
hello-stephen
commented
Nov 28, 2025
Contributor
FE Regression Coverage ReportIncrement line coverage |
924060929 added a commit
to 924060929/incubator-doris
that referenced
this pull request
Dec 1, 2025
Support dereference expression for map/struct/variant
```sql
MySQL root@127.0.0.1:rqg> select
-> named_struct('c', named_struct('c', 100)).c.c,
-> map('a', 10, 'b', 20).a,
-> map('a', 10, 'b', 20).b,
-> cast('{"a":{"b":300}}' as variant).a.b;
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| named_struct('c', named_struct('c', 100)).c.c | map('a', 10, 'b', 20).a | map('a', 10, 'b', 20).b | cast('{"a":{"b":300}}' as variant).a.b |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| 100 | 10 | 20 | 300 |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
```
<b>the slot name parts binding priority:</b>
1. try to bind catalog, if binding failed, jump to 2
2. try to bind database, if binding failed, jump to 3
3. try to bind table, if binding failed, jump to 4
4. try to bind column
for example, `a.a.a.a.a`
first try to bind: catalog=a, database=a, table=a, column=a, sub
fields=a
second try to bind: catalog=<current catalog>, database=a,
table=a, column=a, sub fields=a.a
third try to bind: catalog=<current catalog>, database=<current
database>, table=a, column=a, sub fields=a.a.a
4th try to bind: catalog=<current catalog>, database=<current
database>, table=<underscore table>, column=a, sub
fields=a.a.a.a
(cherry picked from commit ac92fd4)924060929 added a commit
to 924060929/incubator-doris
that referenced
this pull request
Dec 1, 2025
Support dereference expression for map/struct/variant
```sql
MySQL root@127.0.0.1:rqg> select
-> named_struct('c', named_struct('c', 100)).c.c,
-> map('a', 10, 'b', 20).a,
-> map('a', 10, 'b', 20).b,
-> cast('{"a":{"b":300}}' as variant).a.b;
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| named_struct('c', named_struct('c', 100)).c.c | map('a', 10, 'b', 20).a | map('a', 10, 'b', 20).b | cast('{"a":{"b":300}}' as variant).a.b |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| 100 | 10 | 20 | 300 |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
```
<b>the slot name parts binding priority:</b>
1. try to bind catalog, if binding failed, jump to 2
2. try to bind database, if binding failed, jump to 3
3. try to bind table, if binding failed, jump to 4
4. try to bind column
for example, `a.a.a.a.a`
first try to bind: catalog=a, database=a, table=a, column=a, sub
fields=a
second try to bind: catalog=<current catalog>, database=a,
table=a, column=a, sub fields=a.a
third try to bind: catalog=<current catalog>, database=<current
database>, table=a, column=a, sub fields=a.a.a
4th try to bind: catalog=<current catalog>, database=<current
database>, table=<underscore table>, column=a, sub
fields=a.a.a.a
(cherry picked from commit ac92fd4)morrySnow pushed a commit
that referenced
this pull request
Dec 1, 2025
yiguolei pushed a commit
that referenced
this pull request
Dec 2, 2025
16 tasks
924060929 added a commit
that referenced
this pull request
Dec 3, 2025
fix can not prune dereference expression, introduced by #57532
nagisa-kunhah pushed a commit
to nagisa-kunhah/doris
that referenced
this pull request
Dec 14, 2025
Support dereference expression for map/struct/variant
```sql
MySQL root@127.0.0.1:rqg> select
-> named_struct('c', named_struct('c', 100)).c.c,
-> map('a', 10, 'b', 20).a,
-> map('a', 10, 'b', 20).b,
-> cast('{"a":{"b":300}}' as variant).a.b;
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| named_struct('c', named_struct('c', 100)).c.c | map('a', 10, 'b', 20).a | map('a', 10, 'b', 20).b | cast('{"a":{"b":300}}' as variant).a.b |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
| 100 | 10 | 20 | 300 |
+-----------------------------------------------+-------------------------+-------------------------+----------------------------------------+
```
<b>the slot name parts binding priority:</b>
1. try to bind catalog, if binding failed, jump to 2
2. try to bind database, if binding failed, jump to 3
3. try to bind table, if binding failed, jump to 4
4. try to bind column
for example, `a.a.a.a.a`
first try to bind: catalog=a, database=a, table=a, column=a, sub
fields=a
second try to bind: catalog=<current catalog>, database=a,
table=a, column=a, sub fields=a.a
third try to bind: catalog=<current catalog>, database=<current
database>, table=a, column=a, sub fields=a.a.a
4th try to bind: catalog=<current catalog>, database=<current
database>, table=<underscore table>, column=a, sub
fields=a.a.a.anagisa-kunhah pushed a commit
to nagisa-kunhah/doris
that referenced
this pull request
Dec 14, 2025
fix can not prune dereference expression, introduced by apache#57532
924060929 added a commit
that referenced
this pull request
Dec 23, 2025
924060929 added a commit
that referenced
this pull request
Dec 23, 2025
16 tasks
morrySnow pushed a commit
that referenced
this pull request
May 12, 2026
…select view result wrong when view select has variant field (#62907) ### What problem does this PR solve? Related PR: #57532 Problem Summary: `CREATE VIEW` with Nereids could persist wrong SQL when the view definition used dotted `VARIANT` subfield access such as `event_value.video_id`. The buggy persisted SQL replaced the whole subfield expression with the base column `event_value`, so querying the view could return `0` rows while the same CTE queried directly returned `1` row. Reproduction steps: 1. Create an `events` table with `event_value VARIANT<'video_id':largeint, 'duration':bigint>` and `user_connect_info VARIANT<'user_client':text>`. 2. Insert one `watch_time` row where `event_value.video_id = 100`, `event_value.duration = 15000`, and `user_connect_info.user_client = 'ios'`. 3. Create a view with a CTE that projects `CAST(event_value.video_id AS LARGEINT)`, `TRY_CAST(event_value.duration AS INT)`, joins `video_meta`, and aggregates the result. 4. Query the view with `SELECT COUNT(*) FROM v_bug` and `SELECT COUNT(*) FROM v_bug WHERE event_day >= '2026-04-20'`. 5. On a buggy build both view queries return `0`; on the fixed build both return `1`. `SHOW CREATE VIEW` also keeps ``event_value`.`video_id`` instead of losing the subfield path. Root cause: In `ExpressionAnalyzer.bindExpressionByColumn()` and the qualified variants (`bindExpressionByTableColumn()`, `bindExpressionByDbTableColumn()`, `bindExpressionByCatalogDbTableColumn()`), the analyzer attached SQL-index rewrite metadata to the base `SlotReference` before `bindNestedFields()` handled the dotted subfield path. After binding `event_value.video_id` into `ElementAt(SlotReference(event_value), 'video_id')`, the base slot still owned the original SQL span for the whole `event_value.video_id` text. The create-view SQL rewrite later replaced that span with the fully qualified base column only, changing the view definition semantics. | File | Change Description | |------|-------------------| | `fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java` | Delays `addSqlIndexInfo()` until the analyzer knows the reference is an ordinary slot path. For successfully bound nested fields, records a fully qualified dotted replacement that includes the base column and all nested field names. | | `regression-test/suites/ddl_p0/create_view_nereids/test_create_view_variant_nested_field.groovy` | Adds regression coverage for `CREATE VIEW`, `ALTER VIEW`, bracket-style access, and the CTE/join/aggregation shape from the JIRA reproduction. The test queries the created view directly and verifies the row count remains `1`. | Design rationale: This is fixed in expression analysis instead of `BaseViewInfo.SlotDealer` because `ExpressionAnalyzer` is where the dotted name is resolved into a nested-field expression and where the original SQL span is still available with semantic context. `SlotDealer` only sees later slot references during view SQL rewrite, so fixing it there would require guessing whether a base slot originated from a nested-field path. Delaying base-slot rewrite metadata and adding explicit nested-field rewrite metadata keeps ordinary column behavior unchanged while preserving `VARIANT` subfield semantics in view definitions. ### Release note Fixed an issue where `CREATE VIEW` or `ALTER VIEW` could lose dotted `VARIANT` subfield paths in persisted view SQL, causing later queries on the view to return incorrect results.
github-actionsBot
pushed a commit
that referenced
this pull request
May 12, 2026
…select view result wrong when view select has variant field (#62907) ### What problem does this PR solve? Related PR: #57532 Problem Summary: `CREATE VIEW` with Nereids could persist wrong SQL when the view definition used dotted `VARIANT` subfield access such as `event_value.video_id`. The buggy persisted SQL replaced the whole subfield expression with the base column `event_value`, so querying the view could return `0` rows while the same CTE queried directly returned `1` row. Reproduction steps: 1. Create an `events` table with `event_value VARIANT<'video_id':largeint, 'duration':bigint>` and `user_connect_info VARIANT<'user_client':text>`. 2. Insert one `watch_time` row where `event_value.video_id = 100`, `event_value.duration = 15000`, and `user_connect_info.user_client = 'ios'`. 3. Create a view with a CTE that projects `CAST(event_value.video_id AS LARGEINT)`, `TRY_CAST(event_value.duration AS INT)`, joins `video_meta`, and aggregates the result. 4. Query the view with `SELECT COUNT(*) FROM v_bug` and `SELECT COUNT(*) FROM v_bug WHERE event_day >= '2026-04-20'`. 5. On a buggy build both view queries return `0`; on the fixed build both return `1`. `SHOW CREATE VIEW` also keeps ``event_value`.`video_id`` instead of losing the subfield path. Root cause: In `ExpressionAnalyzer.bindExpressionByColumn()` and the qualified variants (`bindExpressionByTableColumn()`, `bindExpressionByDbTableColumn()`, `bindExpressionByCatalogDbTableColumn()`), the analyzer attached SQL-index rewrite metadata to the base `SlotReference` before `bindNestedFields()` handled the dotted subfield path. After binding `event_value.video_id` into `ElementAt(SlotReference(event_value), 'video_id')`, the base slot still owned the original SQL span for the whole `event_value.video_id` text. The create-view SQL rewrite later replaced that span with the fully qualified base column only, changing the view definition semantics. | File | Change Description | |------|-------------------| | `fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java` | Delays `addSqlIndexInfo()` until the analyzer knows the reference is an ordinary slot path. For successfully bound nested fields, records a fully qualified dotted replacement that includes the base column and all nested field names. | | `regression-test/suites/ddl_p0/create_view_nereids/test_create_view_variant_nested_field.groovy` | Adds regression coverage for `CREATE VIEW`, `ALTER VIEW`, bracket-style access, and the CTE/join/aggregation shape from the JIRA reproduction. The test queries the created view directly and verifies the row count remains `1`. | Design rationale: This is fixed in expression analysis instead of `BaseViewInfo.SlotDealer` because `ExpressionAnalyzer` is where the dotted name is resolved into a nested-field expression and where the original SQL span is still available with semantic context. `SlotDealer` only sees later slot references during view SQL rewrite, so fixing it there would require guessing whether a base slot originated from a nested-field path. Delaying base-slot rewrite metadata and adding explicit nested-field rewrite metadata keeps ordinary column behavior unchanged while preserving `VARIANT` subfield semantics in view definitions. ### Release note Fixed an issue where `CREATE VIEW` or `ALTER VIEW` could lose dotted `VARIANT` subfield paths in persisted view SQL, causing later queries on the view to return incorrect results.
zhaorongsheng pushed a commit
to zhaorongsheng/doris
that referenced
this pull request
Jun 4, 2026
…select view result wrong when view select has variant field (apache#62907) ### What problem does this PR solve? Related PR: apache#57532 Problem Summary: `CREATE VIEW` with Nereids could persist wrong SQL when the view definition used dotted `VARIANT` subfield access such as `event_value.video_id`. The buggy persisted SQL replaced the whole subfield expression with the base column `event_value`, so querying the view could return `0` rows while the same CTE queried directly returned `1` row. Reproduction steps: 1. Create an `events` table with `event_value VARIANT<'video_id':largeint, 'duration':bigint>` and `user_connect_info VARIANT<'user_client':text>`. 2. Insert one `watch_time` row where `event_value.video_id = 100`, `event_value.duration = 15000`, and `user_connect_info.user_client = 'ios'`. 3. Create a view with a CTE that projects `CAST(event_value.video_id AS LARGEINT)`, `TRY_CAST(event_value.duration AS INT)`, joins `video_meta`, and aggregates the result. 4. Query the view with `SELECT COUNT(*) FROM v_bug` and `SELECT COUNT(*) FROM v_bug WHERE event_day >= '2026-04-20'`. 5. On a buggy build both view queries return `0`; on the fixed build both return `1`. `SHOW CREATE VIEW` also keeps ``event_value`.`video_id`` instead of losing the subfield path. Root cause: In `ExpressionAnalyzer.bindExpressionByColumn()` and the qualified variants (`bindExpressionByTableColumn()`, `bindExpressionByDbTableColumn()`, `bindExpressionByCatalogDbTableColumn()`), the analyzer attached SQL-index rewrite metadata to the base `SlotReference` before `bindNestedFields()` handled the dotted subfield path. After binding `event_value.video_id` into `ElementAt(SlotReference(event_value), 'video_id')`, the base slot still owned the original SQL span for the whole `event_value.video_id` text. The create-view SQL rewrite later replaced that span with the fully qualified base column only, changing the view definition semantics. | File | Change Description | |------|-------------------| | `fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java` | Delays `addSqlIndexInfo()` until the analyzer knows the reference is an ordinary slot path. For successfully bound nested fields, records a fully qualified dotted replacement that includes the base column and all nested field names. | | `regression-test/suites/ddl_p0/create_view_nereids/test_create_view_variant_nested_field.groovy` | Adds regression coverage for `CREATE VIEW`, `ALTER VIEW`, bracket-style access, and the CTE/join/aggregation shape from the JIRA reproduction. The test queries the created view directly and verifies the row count remains `1`. | Design rationale: This is fixed in expression analysis instead of `BaseViewInfo.SlotDealer` because `ExpressionAnalyzer` is where the dotted name is resolved into a nested-field expression and where the original SQL span is still available with semantic context. `SlotDealer` only sees later slot references during view SQL rewrite, so fixing it there would require guessing whether a base slot originated from a nested-field path. Delaying base-slot rewrite metadata and adding explicit nested-field rewrite metadata keeps ordinary column behavior unchanged while preserving `VARIANT` subfield semantics in view definitions. ### Release note Fixed an issue where `CREATE VIEW` or `ALTER VIEW` could lose dotted `VARIANT` subfield paths in persisted view SQL, causing later queries on the view to return incorrect results.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Support dereference expression for map/struct/variant
the slot name parts binding priority:
for example,
a.a.a.a.afirst try to bind: catalog=a, database=a, table=a, column=a, sub fields=a
second try to bind: catalog=<current catalog>, database=a, table=a, column=a, sub fields=a.a
third try to bind: catalog=<current catalog>, database=<current database>, table=a, column=a, sub fields=a.a.a
4th try to bind: catalog=<current catalog>, database=<current database>, table=<underscore table>, column=a, sub fields=a.a.a.a
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)