[feature](function) add approx_top_sum aggregation function by zzzxl1993 · Pull Request #43643 · apache/doris

zzzxl1993 · 2024-11-11T15:38:13Z

What problem does this PR solve?

Problem Summary:

The function approx_top_sum has been implemented. Here is an example of its usage: select approx_top_sum(c1, c2, c3, 10, 300) from tbl.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-11T15:38:18Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

github-actions

clang-tidy made some suggestions

github-actions · 2024-11-11T15:43:18Z

+
+#pragma once
+
+#include <rapidjson/encodings.h>


warning: 'rapidjson/encodings.h' file not found [clang-diagnostic-error]
#include<rapidjson/encodings.h> ^

zzzxl1993 · 2024-11-12T02:13:02Z

run buildall

zzzxl1993 · 2024-11-12T05:57:30Z

run buildall

zzzxl1993 · 2024-11-12T11:28:42Z

run buildall

superdiaodiao · 2024-11-12T12:06:14Z

- qt_sql """ select approx_top_k(clientip) from ${tableName}; """
- qt_sql """ select approx_top_k(clientip, 10) from ${tableName}; """


May I ask why deleting these tests in a feature PR?

Currently, it is difficult to implement default values for the variable arguments in the function framework. It is hoped that users will directly specify the last two parameters.

zzzxl1993 · 2024-11-12T13:34:48Z

run buildall

xiaokang · 2024-11-13T23:21:46Z

 struct AggregateFunctionAttr {
 bool enable_decimal256 {false};
- std::vector<std::pair<std::string, bool>> column_infos;
+ std::vector<std::string> column_names;


Why change this basic data structure?

Optimize the redundant data structures added in approx_top_k.

xiaokang · 2024-11-13T23:35:31Z

+ sub_writer.Key(_column_names[i].data(), _column_names[i].size());
+ sub_writer.String(row_str.data(), row_str.size());
+ }
+ sub_writer.Key("count");


xiaokang · 2024-11-13T23:36:56Z

 @Override
 public void checkLegalityBeforeTypeCoercion() {
- if (arity() < 1) {
+ if (arity() < 3) {


Why change arguments num from 1 to 3?

The existing function framework code cannot implement approx_top_k and approx_top_sum with default values.

zzzxl1993 · 2024-11-14T15:39:51Z

run buildall

doris-robot · 2024-11-14T17:23:27Z

TeamCity be ut coverage result:
Function Coverage: 37.96% (9904/26090)
Line Coverage: 29.16% (82764/283794)
Region Coverage: 28.29% (42514/150280)
Branch Coverage: 24.86% (21556/86694)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cf48a1bd8a8007fe40606b7a4d907a182403e85c_cf48a1bd8a8007fe40606b7a4d907a182403e85c/report/index.html

zzzxl1993 · 2024-11-15T03:34:45Z

run buildall

doris-robot · 2024-11-15T04:15:31Z

TeamCity be ut coverage result:
Function Coverage: 37.96% (9903/26090)
Line Coverage: 29.16% (82762/283795)
Region Coverage: 28.29% (42519/150280)
Branch Coverage: 24.86% (21554/86694)
Coverage Report: http://coverage.selectdb-in.cc/coverage/adf71e19982471db714eaf0aa221f815962c102d_adf71e19982471db714eaf0aa221f815962c102d/report/index.html

xiaokang

LGTM

github-actions · 2024-11-17T01:17:24Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-17T01:17:26Z

PR approved by anyone and no changes requested.

qidaye

LGTM

…3643) Problem Summary: 1. The function approx_top_sum has been implemented. Here is an example of its usage: select approx_top_sum(c1, c2, c3, 10, 300) from tbl. ### Release note Add new function `approx_top_sum`.

…3643) Problem Summary: 1. The function approx_top_sum has been implemented. Here is an example of its usage: select approx_top_sum(c1, c2, c3, 10, 300) from tbl. Add new function `approx_top_sum`.

pick #40813 pick #43643

…3643) Problem Summary: 1. The function approx_top_sum has been implemented. Here is an example of its usage: select approx_top_sum(c1, c2, c3, 10, 300) from tbl. ### Release note Add new function `approx_top_sum`.

github-actionsBot reviewed Nov 11, 2024
View reviewed changes

zzzxl1993 force-pushed the 202411111526 branch from ca8d282 to 1ea36eeCompare November 12, 2024 05:57

zzzxl1993 force-pushed the 202411111526 branch from 1ea36ee to 17ecc13Compare November 12, 2024 11:28

superdiaodiao reviewed Nov 12, 2024
View reviewed changes

zzzxl1993 force-pushed the 202411111526 branch from 17ecc13 to 23d7adcCompare November 12, 2024 13:34

xiaokang reviewed Nov 13, 2024
View reviewed changes

zzzxl1993 force-pushed the 202411111526 branch from 23d7adc to cf48a1bCompare November 14, 2024 15:39

[feature](function) add approx_top_sum aggregation function
adf71e1

zzzxl1993 force-pushed the 202411111526 branch from cf48a1b to adf71e1Compare November 15, 2024 03:34

xiaokang approved these changes Nov 17, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 17, 2024

github-actionsBot added the reviewed label Nov 17, 2024

qidaye approved these changes Nov 17, 2024
View reviewed changes

qidaye merged commit 13ced20 into apache:masterNov 17, 2024

zzzxl1993 mentioned this pull request Nov 17, 2024
[feature](function) add approx_top_k aggregation function #44082
Merged

qidaye pushed a commit that referenced this pull request Nov 18, 2024
[feature](function) add approx_top_k aggregation function (#44082)
15e26de
pick #40813 pick #43643

		qt_sql """ select approx_top_k(clientip) from ${tableName}; """
		qt_sql """ select approx_top_k(clientip, 10) from ${tableName}; """


		#pragma once

		#include <rapidjson/encodings.h>

Uh oh!

Conversation

zzzxl1993 commented Nov 11, 2024• edited by qidaye Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 11, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBotNov 11, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993 commented Nov 12, 2024

Uh oh!

zzzxl1993 commented Nov 12, 2024

Uh oh!

zzzxl1993 commented Nov 12, 2024

Uh oh!

superdiaodiaoNov 12, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zzzxl1993Nov 12, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993 commented Nov 12, 2024

Uh oh!

xiaokangNov 13, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

xiaokangNov 13, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

xiaokangNov 13, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

zzzxl1993 commented Nov 14, 2024

Uh oh!

doris-robot commented Nov 14, 2024

Uh oh!

zzzxl1993 commented Nov 15, 2024

Uh oh!

doris-robot commented Nov 15, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Nov 17, 2024

Uh oh!

github-actionsBot commented Nov 17, 2024

Uh oh!

qidaye left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zzzxl1993 commented Nov 11, 2024•
edited by qidaye
Loading

superdiaodiaoNov 12, 2024•
edited
Loading