Uh oh!
There was an error while loading. Please reload this page.
[Feat](nereids) add transform rule MergePercentileToArray - #34313
Merged
Conversation
doris-robot
commented
Apr 29, 2024
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
feiniaofeiafei
commented
Apr 29, 2024
CollaboratorAuthor
run buildall |
doris-robot
commented
Apr 29, 2024
TPC-DS: Total hot run time: 187124 ms |
morrySnow
marked this pull request as draft
May 6, 2024 02:40
morrySnow
commented
May 6, 2024
Contributor
wait for percentileArray perf opt |
feiniaofeiafei
marked this pull request as ready for review
May 30, 2024 08:03
feiniaofeiafeiforce-pushed
the
MergePercentileToArray
branch
from
May 30, 2024 08:03
e97e3ff to
64401c7Comparefeiniaofeiafei
commented
May 30, 2024
CollaboratorAuthor
run buildall |
feiniaofeiafeiforce-pushed
the
MergePercentileToArray
branch
from
May 31, 2024 08:08
64401c7 to
690d4d1Comparefeiniaofeiafei
commented
May 31, 2024
CollaboratorAuthor
run buildall |
doris-robot
commented
May 31, 2024
TPC-H: Total hot run time: 41112 ms |
feiniaofeiafei
commented
May 31, 2024
CollaboratorAuthor
run xcloud_p1 |
morrySnow
commented
Jun 3, 2024
Contributor
run cloud_p1 |
morrySnow
commented
Jun 3, 2024
Contributor
run performance |
doris-robot
commented
Jun 3, 2024
TPC-H: Total hot run time: 41695 ms |
doris-robot
commented
Jun 3, 2024
TPC-DS: Total hot run time: 169184 ms |
doris-robot
commented
Jun 3, 2024
ClickBench: Total hot run time: 30.65 s |
morrySnow
approved these changes
Jun 3, 2024
Contributor
PR approved by at least one committer and no changes requested. |
Contributor
PR approved by anyone and no changes requested. |
924060929
approved these changes
Jun 3, 2024
feiniaofeiafei added a commit
to feiniaofeiafei/doris
that referenced
this pull request
Jun 3, 2024
MergePercentileToArray is to perform a transformation in this case: select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk; ===> select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;
dataroaring pushed a commit
that referenced
this pull request
Jun 4, 2024
MergePercentileToArray is to perform a transformation in this case: select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk; ===> select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;
morrySnow pushed a commit
that referenced
this pull request
Jun 4, 2024
cherry-pick #34313 to branch-2.1 MergePercentileToArray is to perform a transformation in this case: select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk; ==> select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;
seawinde pushed a commit
to seawinde/doris
that referenced
this pull request
Jun 5, 2024
MergePercentileToArray is to perform a transformation in this case: select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk; ===> select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;
16 tasks
starocean999 pushed a commit
that referenced
this pull request
Dec 2, 2024
…on (#44783) Related PR: #34313 Problem Summary The original PR did not handle the following scenario: ```sql SELECT SUM(a), PERCENTILE(pk, 0.1) AS c1, PERCENTILE(pk, 0.1) AS c2, PERCENTILE(pk, 0.4) AS c3 FROM test_merge_percentile; ``` In this case, the aggregate outputs include two identical functions (PERCENTILE(pk, 0.1)). When constructing the LogicalProject, a map was used where the key is the child of an Alias and the value is the Alias itself. However, this approach loses information when two Aliases share the same child. This PR modifies the map structure to use the child of an Alias as the key and a list of Alias objects as the value. This ensures that all Alias instances with the same child are preserved, resolving the issue of lost information in such cases.
github-actionsBot
pushed a commit
that referenced
this pull request
Dec 2, 2024
…on (#44783) Related PR: #34313 Problem Summary The original PR did not handle the following scenario: ```sql SELECT SUM(a), PERCENTILE(pk, 0.1) AS c1, PERCENTILE(pk, 0.1) AS c2, PERCENTILE(pk, 0.4) AS c3 FROM test_merge_percentile; ``` In this case, the aggregate outputs include two identical functions (PERCENTILE(pk, 0.1)). When constructing the LogicalProject, a map was used where the key is the child of an Alias and the value is the Alias itself. However, this approach loses information when two Aliases share the same child. This PR modifies the map structure to use the child of an Alias as the key and a list of Alias objects as the value. This ensures that all Alias instances with the same child are preserved, resolving the issue of lost information in such cases.
github-actionsBot
pushed a commit
that referenced
this pull request
Dec 2, 2024
…on (#44783) Related PR: #34313 Problem Summary The original PR did not handle the following scenario: ```sql SELECT SUM(a), PERCENTILE(pk, 0.1) AS c1, PERCENTILE(pk, 0.1) AS c2, PERCENTILE(pk, 0.4) AS c3 FROM test_merge_percentile; ``` In this case, the aggregate outputs include two identical functions (PERCENTILE(pk, 0.1)). When constructing the LogicalProject, a map was used where the key is the child of an Alias and the value is the Alias itself. However, this approach loses information when two Aliases share the same child. This PR modifies the map structure to use the child of an Alias as the key and a list of Alias objects as the value. This ensures that all Alias instances with the same child are preserved, resolving the issue of lost information in such cases.
16 tasks
morrySnow pushed a commit
that referenced
this pull request
May 6, 2025
### What problem does this PR solve? Related PR: #34313 Problem Summary: When the second argument of percentile is not literal, will report error ：index out of range. This pr fix this bug, taking the second argument of percentile directly, and changing the arguments of percentilearray from array literal to array expression.
github-actionsBot
pushed a commit
that referenced
this pull request
May 6, 2025
### What problem does this PR solve? Related PR: #34313 Problem Summary: When the second argument of percentile is not literal, will report error ：index out of range. This pr fix this bug, taking the second argument of percentile directly, and changing the arguments of percentilearray from array literal to array expression.
github-actionsBot
pushed a commit
that referenced
this pull request
May 6, 2025
### What problem does this PR solve? Related PR: #34313 Problem Summary: When the second argument of percentile is not literal, will report error ：index out of range. This pr fix this bug, taking the second argument of percentile directly, and changing the arguments of percentilearray from array literal to array expression.
koarz pushed a commit
to koarz/doris
that referenced
this pull request
Jun 4, 2025
### What problem does this PR solve? Related PR: apache#34313 Problem Summary: When the second argument of percentile is not literal, will report error ：index out of range. This pr fix this bug, taking the second argument of percentile directly, and changing the arguments of percentilearray from array literal to array expression.
HappenLee pushed a commit
to HappenLee/incubator-doris
that referenced
this pull request
Apr 24, 2026
MergePercentileToArray is to perform a transformation in this case: select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk; ===> select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;
HappenLee pushed a commit
to HappenLee/incubator-doris
that referenced
this pull request
Apr 24, 2026
…on (apache#44783) Related PR: apache#34313 Problem Summary The original PR did not handle the following scenario: ```sql SELECT SUM(a), PERCENTILE(pk, 0.1) AS c1, PERCENTILE(pk, 0.1) AS c2, PERCENTILE(pk, 0.4) AS c3 FROM test_merge_percentile; ``` In this case, the aggregate outputs include two identical functions (PERCENTILE(pk, 0.1)). When constructing the LogicalProject, a map was used where the key is the child of an Alias and the value is the Alias itself. However, this approach loses information when two Aliases share the same child. This PR modifies the map structure to use the child of an Alias as the key and a list of Alias objects as the value. This ensures that all Alias instances with the same child are preserved, resolving the issue of lost information in such cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MergePercentileToArray is to perform a transformation in this case:
select ss_item_sk, percentile(ss_quantity,0.9), percentile(ss_quantity,0.6), percentile(ss_quantity,0.3) from store_sales group by ss_item_sk;
=》
select ss_item_sk, percentile_array(ss_quantity,[0.3,0.6,0.9]) from store_sales group by ss_item_sk;