Skip to content

[Feature](func) Support function MAKE_SET - #56367

Merged
zclllyybb merged 3 commits into
apache:masterfrom
linrrzqqq:make_set
Sep 25, 2025
Merged

[Feature](func) Support function MAKE_SET#56367
zclllyybb merged 3 commits into
apache:masterfrom
linrrzqqq:make_set

Conversation

@linrrzqqq

@linrrzqqqlinrrzqqq commented Sep 23, 2025•
edited
Loading

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: #48203

Related PR: #xxx

Problem Summary:

Release note

MAKE_SET(bit, str1, str2...)
Returns a set value (a string containing substrings separated by , characters) consisting of the strings that have the corresponding bit in bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL values in str1, str2, ... are not appended to the result.

mysql> SELECT MAKE_SET(2, 'hello', 'goodbye', 'world');
+------------------------------------------+
| MAKE_SET(2, 'hello', 'goodbye', 'world') |
+------------------------------------------+
| goodbye |
+------------------------------------------+
mysql> SELECT MAKE_SET(4, 'a', 'b', NULL);
+-----------------------------+
| MAKE_SET(4, 'a', 'b', NULL) |
+-----------------------------+
| |
+-----------------------------+
mysql> SELECT MAKE_SET(NULL, 'a', 'b', 'c');
+-------------------------------+
| MAKE_SET(NULL, 'a', 'b', 'c') |
+-------------------------------+
| NULL |
+-------------------------------+

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@linrrzqqq

linrrzqqq commented Sep 23, 2025

Copy link
Copy Markdown
CollaboratorAuthor

run buildall

@doris-robot

doris-robot commented Sep 23, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.49 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98109ce0169cf2ce5407a08c2154299c1d15657c, data reload: false
query1	0.05	0.04	0.05
query2	0.09	0.06	0.06
query3	0.25	0.09	0.08
query4	1.62	0.11	0.12
query5	0.29	0.27	0.25
query6	1.17	0.65	0.64
query7	0.03	0.02	0.02
query8	0.05	0.05	0.04
query9	0.62	0.52	0.52
query10	0.58	0.58	0.58
query11	0.16	0.15	0.11
query12	0.15	0.12	0.12
query13	0.63	0.63	0.62
query14	1.04	1.06	1.03
query15	0.87	0.88	0.87
query16	0.40	0.40	0.39
query17	1.05	1.09	1.07
query18	0.21	0.20	0.20
query19	2.24	2.02	2.09
query20	0.02	0.01	0.02
query21	15.56	0.95	0.58
query22	0.76	1.21	0.76
query23	14.82	1.38	0.65
query24	7.11	1.63	0.46
query25	0.45	0.16	0.23
query26	0.67	0.16	0.14
query27	0.08	0.05	0.06
query28	9.46	1.37	0.94
query29	12.58	3.91	3.22
query30	0.29	0.13	0.12
query31	2.82	0.60	0.40
query32	3.27	0.56	0.47
query33	3.13	3.12	3.12
query34	16.19	5.48	4.86
query35	4.91	4.94	4.89
query36	0.72	0.51	0.49
query37	0.10	0.08	0.08
query38	0.08	0.05	0.05
query39	0.03	0.03	0.03
query40	0.17	0.15	0.15
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.88 s
Total hot run time: 30.49 s

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 8.00% (2/25) 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Sep 23, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 3.75% (3/80) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.40% (17643/33670)
Line Coverage37.63% (160172/425646)
Region Coverage32.16% (121970/379296)
Branch Coverage33.50% (53466/159614)

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 81.25% (65/80) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.14% (23397/32888)
Line Coverage57.56% (244359/424540)
Region Coverage52.88% (203257/384348)
Branch Coverage54.56% (87447/160267)

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 92.00% (23/25) 🎉
Increment coverage report
Complete coverage report

zclllyybb
zclllyybb reviewed Sep 24, 2025
View reviewed changes
Comment threadbe/src/vec/functions/function_string.h Outdated
Comment threadbe/src/vec/functions/function_string.h Outdated
Comment threadbe/src/vec/functions/function_string.h Outdated
Comment threadregression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
Comment thread...-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/MakeSet.java
@linrrzqqq

linrrzqqq commented Sep 24, 2025

Copy link
Copy Markdown
CollaboratorAuthor

run buildall

@doris-robot

doris-robot commented Sep 24, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e2eec0d7fa153fe0bb20f86790cbd260b1df26d2, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.06	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.29	0.26	0.25
query6	1.19	0.66	0.64
query7	0.04	0.03	0.03
query8	0.06	0.05	0.04
query9	0.62	0.53	0.52
query10	0.58	0.60	0.59
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.66	0.62
query14	1.06	1.03	1.03
query15	0.88	0.88	0.84
query16	0.40	0.42	0.38
query17	1.05	1.02	1.03
query18	0.22	0.21	0.19
query19	2.25	2.10	2.13
query20	0.02	0.02	0.01
query21	15.43	0.96	0.57
query22	0.77	1.24	0.79
query23	14.77	1.40	0.67
query24	7.68	1.36	0.36
query25	0.30	0.11	0.20
query26	0.67	0.17	0.13
query27	0.07	0.06	0.06
query28	9.12	1.33	0.93
query29	12.54	3.90	3.28
query30	0.27	0.14	0.13
query31	2.84	0.61	0.39
query32	3.24	0.55	0.47
query33	3.06	3.05	3.09
query34	16.14	5.47	4.81
query35	4.89	4.96	4.96
query36	0.69	0.52	0.50
query37	0.10	0.08	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.04
query40	0.19	0.16	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 104.67 s
Total hot run time: 30.39 s

@linrrzqqq

linrrzqqq commented Sep 24, 2025

Copy link
Copy Markdown
CollaboratorAuthor

run buildall

@doris-robot

doris-robot commented Sep 24, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4124ef3d8eeae54b7ea7cde6d1232b5bd1e3fe8, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.06	0.06
query3	0.27	0.08	0.08
query4	1.60	0.12	0.12
query5	0.28	0.26	0.25
query6	1.18	0.66	0.64
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.63	0.54	0.52
query10	0.58	0.57	0.58
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.63	0.63	0.63
query14	1.01	1.02	1.03
query15	0.87	0.85	0.88
query16	0.41	0.40	0.40
query17	1.05	1.08	1.04
query18	0.22	0.20	0.20
query19	1.93	1.82	1.85
query20	0.02	0.02	0.01
query21	15.44	0.93	0.57
query22	0.75	1.34	0.74
query23	14.74	1.40	0.67
query24	6.47	1.81	1.01
query25	0.46	0.22	0.07
query26	0.52	0.17	0.14
query27	0.07	0.06	0.06
query28	10.18	1.38	0.93
query29	12.59	3.98	3.23
query30	0.29	0.14	0.13
query31	2.86	0.60	0.40
query32	3.24	0.57	0.47
query33	3.11	3.14	3.11
query34	16.17	5.46	4.84
query35	4.99	4.94	4.90
query36	0.69	0.53	0.51
query37	0.12	0.08	0.08
query38	0.07	0.05	0.05
query39	0.04	0.04	0.03
query40	0.17	0.16	0.14
query41	0.09	0.03	0.04
query42	0.04	0.04	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.37 s
Total hot run time: 30.72 s

@hello-stephen

hello-stephen commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 3.70% (3/81) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.39% (17611/33613)
Line Coverage37.61% (159960/425317)
Region Coverage32.14% (121921/379365)
Branch Coverage33.50% (53493/159678)

@hello-stephen

hello-stephen commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 81.48% (66/81) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.12% (23339/32817)
Line Coverage57.54% (244077/424162)
Region Coverage52.91% (203400/384396)
Branch Coverage54.56% (87471/160327)

@hello-stephen

hello-stephen commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 88.46% (23/26) 🎉
Increment coverage report
Complete coverage report

@linrrzqqq

linrrzqqq commented Sep 24, 2025

Copy link
Copy Markdown
CollaboratorAuthor

run p0

@hello-stephen

hello-stephen commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 81.48% (66/81) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.13% (23342/32817)
Line Coverage57.55% (244105/424162)
Region Coverage52.91% (203387/384396)
Branch Coverage54.56% (87471/160327)

@hello-stephen

hello-stephen commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 88.46% (23/26) 🎉
Increment coverage report
Complete coverage report

zclllyybb
zclllyybb approved these changes Sep 25, 2025
View reviewed changes

@zclllyybbzclllyybb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 25, 2025
@github-actions

github-actionsBot commented Sep 25, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Sep 25, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Sep 25, 2025
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zclllyybb
zclllyybb merged commit 9f9f1eb into apache:masterSep 25, 2025
27 of 29 checks passed
@linrrzqqq
linrrzqqq deleted the make_set branch September 25, 2025 07:18
github-actionsBot pushed a commit that referenced this pull request Sep 27, 2025
@linrrzqqqYour Name
[Feature](func) Support function MAKE_SET (#56367)
34321c1
Issue Number: #48203 `MAKE_SET(bit, str1, str2...)`
Returns a set value (a string containing substrings separated by ,
characters) consisting of the strings that have the corresponding bit in
bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL
values in str1, str2, ... are not appended to the result.
```text
mysql> SELECT MAKE_SET(2, 'hello', 'goodbye', 'world');
+------------------------------------------+
| MAKE_SET(2, 'hello', 'goodbye', 'world') |
+------------------------------------------+
| goodbye |
+------------------------------------------+
mysql> SELECT MAKE_SET(4, 'a', 'b', NULL);
+-----------------------------+
| MAKE_SET(4, 'a', 'b', NULL) |
+-----------------------------+
| |
+-----------------------------+
mysql> SELECT MAKE_SET(NULL, 'a', 'b', 'c');
+-------------------------------+
| MAKE_SET(NULL, 'a', 'b', 'c') |
+-------------------------------+
| NULL |
+-------------------------------+
```
@github-actionsgithub-actionsBot mentioned this pull request Sep 27, 2025
Merged
yiguolei pushed a commit that referenced this pull request Sep 27, 2025
@github-actions@linrrzqqq
branch-4.0: [Feature](func) Support function MAKE_SET#56367(#56524)
31bec06
Cherry-picked from #56367
Co-authored-by: linrrarity <linzhenqi@selectdb.com>
@morrySnowmorrySnow mentioned this pull request Jun 29, 2026
Merged
morrySnow pushed a commit that referenced this pull request Jun 29, 2026
@linrrzqqq
[Fix](fold_const) MAKE_SET constant folding should clear (#64907)
9ec9d5f
related PR: #56367
Problem Summary:
`MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions
in the FE constant folding path, which leads to incorrect clearing of
high bits when `pos >= 32` due to integer shift modulo.
For Java `int` shifts, the shift distance is masked with 0x1F, which
means only the low 5 bits are used:
- `pos = 0..31` -> normal
- `pos = 32` -> treated as `0`
- `pos = 33` -> treated as `1`
- `pos = 64` -> treated as `0` again
For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32)
expect: `bit &= ~(1 << 32)`
got: `bit &= ~(1 << 0)`
That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32
- the loop never makes progress
- the same string is appended again and again
- Java throws OutOfMemoryError: Java heap space
before(FE constant folding failed or FE OOM):
```text
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(11) |
| constant exprs: |
| make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') |
| |
| |
| |
| ========== STATISTICS ========== |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
17 rows in set (28.822 sec)
```
```text
2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a
18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail
ed.
org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei
ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time:
{"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra
nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1}
```
now:
```text
Doris> SET debug_skip_fold_constant = 0;
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+----------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(12) |
| constant exprs: |
| 'a32' |
| |
| |
| |
| ========== STATISTICS ========== |
+----------------------------------+
```
github-actionsBot pushed a commit that referenced this pull request Jun 29, 2026
@linrrzqqqYour Name
[Fix](fold_const) MAKE_SET constant folding should clear (#64907)
2db9d09
related PR: #56367
Problem Summary:
`MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions
in the FE constant folding path, which leads to incorrect clearing of
high bits when `pos >= 32` due to integer shift modulo.
For Java `int` shifts, the shift distance is masked with 0x1F, which
means only the low 5 bits are used:
- `pos = 0..31` -> normal
- `pos = 32` -> treated as `0`
- `pos = 33` -> treated as `1`
- `pos = 64` -> treated as `0` again
For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32)
expect: `bit &= ~(1 << 32)`
got: `bit &= ~(1 << 0)`
That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32
- the loop never makes progress
- the same string is appended again and again
- Java throws OutOfMemoryError: Java heap space
before(FE constant folding failed or FE OOM):
```text
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(11) |
| constant exprs: |
| make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') |
| |
| |
| |
| ========== STATISTICS ========== |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
17 rows in set (28.822 sec)
```
```text
2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a
18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail
ed.
org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei
ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time:
{"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra
nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1}
```
now:
```text
Doris> SET debug_skip_fold_constant = 0;
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+----------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(12) |
| constant exprs: |
| 'a32' |
| |
| |
| |
| ========== STATISTICS ========== |
+----------------------------------+
```
github-actionsBot pushed a commit that referenced this pull request Jun 29, 2026
@linrrzqqqYour Name
[Fix](fold_const) MAKE_SET constant folding should clear (#64907)
eef6d5b
related PR: #56367
Problem Summary:
`MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions
in the FE constant folding path, which leads to incorrect clearing of
high bits when `pos >= 32` due to integer shift modulo.
For Java `int` shifts, the shift distance is masked with 0x1F, which
means only the low 5 bits are used:
- `pos = 0..31` -> normal
- `pos = 32` -> treated as `0`
- `pos = 33` -> treated as `1`
- `pos = 64` -> treated as `0` again
For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32)
expect: `bit &= ~(1 << 32)`
got: `bit &= ~(1 << 0)`
That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32
- the loop never makes progress
- the same string is appended again and again
- Java throws OutOfMemoryError: Java heap space
before(FE constant folding failed or FE OOM):
```text
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(11) |
| constant exprs: |
| make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') |
| |
| |
| |
| ========== STATISTICS ========== |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
17 rows in set (28.822 sec)
```
```text
2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a
18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail
ed.
org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei
ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time:
{"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra
nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1}
```
now:
```text
Doris> SET debug_skip_fold_constant = 0;
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+----------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(12) |
| constant exprs: |
| 'a32' |
| |
| |
| |
| ========== STATISTICS ========== |
+----------------------------------+
```
morrySnow pushed a commit that referenced this pull request Aug 31, 2026
@linrrzqqq@morrySnow
[Fix](fold_const) MAKE_SET constant folding should clear (#64907)
f34ec15
related PR: #56367
Problem Summary:
`MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions
in the FE constant folding path, which leads to incorrect clearing of
high bits when `pos >= 32` due to integer shift modulo.
For Java `int` shifts, the shift distance is masked with 0x1F, which
means only the low 5 bits are used:
- `pos = 0..31` -> normal
- `pos = 32` -> treated as `0`
- `pos = 33` -> treated as `1`
- `pos = 64` -> treated as `0` again
For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32)
expect: `bit &= ~(1 << 32)`
got: `bit &= ~(1 << 0)`
That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32
- the loop never makes progress
- the same string is appended again and again
- Java throws OutOfMemoryError: Java heap space
before(FE constant folding failed or FE OOM):
```text
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(11) |
| constant exprs: |
| make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') |
| |
| |
| |
| ========== STATISTICS ========== |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
17 rows in set (28.822 sec)
```
```text
2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a
18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail
ed.
org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei
ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time:
{"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra
nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1}
```
now:
```text
Doris> SET debug_skip_fold_constant = 0;
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+----------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(12) |
| constant exprs: |
| 'a32' |
| |
| |
| |
| ========== STATISTICS ========== |
+----------------------------------+
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLeeHappenLeeHappenLee approved these changes
@zclllyybbzclllyybbzclllyybb approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.0-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@linrrzqqq@hello-stephen@doris-robot@HappenLee@zclllyybb@yiguolei