[Bug](aggregate) fix bitmap_union return error result in query sql by zhangstar333 · Pull Request #52033 · apache/doris

zhangstar333 · 2025-06-20T05:24:30Z

What problem does this PR solve?

Problem Summary:
the bitmap union maybe could call add_batch firstly, and the call add function,
so if add_batch function not update is_first variable, it's will be loss data in union.

 static void add(BitmapValue& res, const BitmapValue& data, bool& is_first) {
if (UNLIKELY(is_first)) {
res = data;
is_first = false;
} else {
res |= data;
}
}
static void add_batch(BitmapValue& res, std::vector<const BitmapValue*>& data, bool& is_first) {
res.fastunion(data);
is_first = false; // before not set this
}

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-06-20T05:24:35Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zhangstar333 · 2025-06-20T05:27:36Z

run buildall

doris-robot · 2025-06-20T05:56:40Z

TPC-H: Total hot run time: 34060 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b59f9a1a320d3e7d2435b277677e6c8435bd449b, data reload: false
------ Round 1 ----------------------------------
q1	17632	5334	4974	4974
q2	1953	290	182	182
q3	10358	1341	728	728
q4	10309	1008	549	549
q5	8767	2433	2338	2338
q6	195	166	136	136
q7	925	746	629	629
q8	9330	1295	1158	1158
q9	6755	5132	5150	5132
q10	6949	2393	1987	1987
q11	502	297	273	273
q12	351	354	229	229
q13	17792	3685	3057	3057
q14	231	231	217	217
q15	568	490	470	470
q16	442	442	375	375
q17	609	893	387	387
q18	7689	7138	7069	7069
q19	1474	957	589	589
q20	355	346	227	227
q21	3865	3168	2408	2408
q22	1020	982	946	946
Total cold run time: 108071 ms
Total hot run time: 34060 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5182	5128	5125	5125
q2	241	323	225	225
q3	2123	2656	2331	2331
q4	1417	1828	1315	1315
q5	4249	4293	4415	4293
q6	218	173	132	132
q7	2016	1914	1729	1729
q8	2591	2758	2545	2545
q9	7099	7144	7113	7113
q10	3080	3257	2810	2810
q11	578	495	498	495
q12	704	778	629	629
q13	3560	3901	3310	3310
q14	297	303	289	289
q15	518	471	470	470
q16	480	501	462	462
q17	1170	1611	1319	1319
q18	7346	7232	7168	7168
q19	839	861	913	861
q20	1918	2049	1789	1789
q21	4874	4444	4313	4313
q22	1057	991	947	947
Total cold run time: 51557 ms
Total hot run time: 49670 ms

doris-robot · 2025-06-20T06:08:10Z

TPC-DS: Total hot run time: 185134 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b59f9a1a320d3e7d2435b277677e6c8435bd449b, data reload: false
query1	960	406	393	393
query2	6522	1797	1815	1797
query3	6751	222	222	222
query4	26144	23401	22906	22906
query5	4341	645	496	496
query6	326	225	214	214
query7	4623	501	290	290
query8	291	238	219	219
query9	8611	2634	2634	2634
query10	476	356	270	270
query11	15416	15100	14815	14815
query12	167	109	111	109
query13	1658	535	426	426
query14	8852	6485	6317	6317
query15	205	203	168	168
query16	7338	641	481	481
query17	1214	719	597	597
query18	2027	413	308	308
query19	195	219	170	170
query20	120	118	121	118
query21	218	135	125	125
query22	4059	4105	4024	4024
query23	33890	32986	33036	32986
query24	8424	2395	2407	2395
query25	536	469	404	404
query26	934	268	153	153
query27	2722	520	344	344
query28	4297	2126	2116	2116
query29	731	567	450	450
query30	287	224	190	190
query31	905	804	746	746
query32	74	66	62	62
query33	546	363	330	330
query34	796	847	552	552
query35	797	834	732	732
query36	948	995	891	891
query37	118	104	76	76
query38	4099	4114	4063	4063
query39	1494	1439	1409	1409
query40	210	120	108	108
query41	70	57	60	57
query42	128	113	108	108
query43	482	506	483	483
query44	1337	826	827	826
query45	184	176	168	168
query46	863	1035	639	639
query47	1727	1786	1686	1686
query48	388	443	322	322
query49	709	490	393	393
query50	675	709	415	415
query51	4136	4180	4155	4155
query52	114	110	106	106
query53	242	270	183	183
query54	587	578	506	506
query55	96	90	88	88
query56	325	320	299	299
query57	1180	1203	1111	1111
query58	262	275	268	268
query59	2617	2687	2536	2536
query60	341	328	301	301
query61	125	122	135	122
query62	806	708	661	661
query63	226	199	192	192
query64	3606	1021	698	698
query65	4279	4158	4240	4158
query66	1032	422	319	319
query67	15733	15634	15112	15112
query68	8670	904	542	542
query69	493	308	292	292
query70	1224	1131	1126	1126
query71	488	335	306	306
query72	5527	4630	4599	4599
query73	691	585	353	353
query74	9116	8987	8620	8620
query75	4013	3200	2725	2725
query76	3619	1201	768	768
query77	779	386	306	306
query78	9941	10229	9325	9325
query79	1938	824	608	608
query80	611	511	450	450
query81	476	254	222	222
query82	424	133	102	102
query83	292	265	230	230
query84	292	105	90	90
query85	774	424	327	327
query86	334	305	302	302
query87	4507	4596	4313	4313
query88	2918	2270	2270	2270
query89	382	324	286	286
query90	1911	217	214	214
query91	144	144	112	112
query92	76	65	57	57
query93	1115	921	595	595
query94	680	420	273	273
query95	379	295	280	280
query96	486	565	282	282
query97	2720	2779	2647	2647
query98	227	224	203	203
query99	1400	1377	1291	1291
Total cold run time: 271239 ms
Total hot run time: 185134 ms

doris-robot · 2025-06-20T06:13:25Z

ClickBench: Total hot run time: 29.22 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b59f9a1a320d3e7d2435b277677e6c8435bd449b, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.45	0.40	0.42
query6	1.15	0.67	0.67
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.57	0.52	0.51
query10	0.58	0.56	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.65	0.61	0.62
query14	0.80	0.83	0.81
query15	0.90	0.88	0.87
query16	0.39	0.38	0.40
query17	1.08	1.05	1.05
query18	0.22	0.21	0.20
query19	2.03	1.88	1.88
query20	0.01	0.01	0.01
query21	15.41	0.89	0.54
query22	0.74	1.24	0.66
query23	14.90	1.38	0.64
query24	7.84	0.80	0.55
query25	0.50	0.13	0.11
query26	0.66	0.16	0.13
query27	0.06	0.06	0.05
query28	9.12	0.89	0.44
query29	12.68	4.01	3.31
query30	0.25	0.09	0.07
query31	2.83	0.62	0.39
query32	3.23	0.55	0.46
query33	3.00	3.07	3.08
query34	16.10	5.42	4.73
query35	4.81	4.81	4.82
query36	0.70	0.52	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.03
query39	0.03	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.04
Total cold run time: 104.48 s
Total hot run time: 29.22 s

HappenLee

LGTM

github-actions · 2025-06-20T10:26:36Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-06-20T10:26:38Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-06-23T08:25:52Z

BE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	56.61% (15117/26702)
Line Coverage	45.39% (135457/298424)
Region Coverage	44.49% (68094/153039)
Branch Coverage	39.06% (34957/89502)

hello-stephen · 2025-06-23T08:35:36Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	61.36% (16119/26268)
Line Coverage	50.75% (151313/298143)
Region Coverage	48.19% (86683/179860)
Branch Coverage	41.46% (42360/102170)

BiteTheDDDDt

LGTM

…pache#52033) Problem Summary: the bitmap union maybe could call add_batch firstly, and the call add function, so if add_batch function not update is_first variable, it's will be loss data in union. ``` static void add(BitmapValue& res, const BitmapValue& data, bool& is_first) { if (UNLIKELY(is_first)) { res = data; is_first = false; } else { res |= data; } } static void add_batch(BitmapValue& res, std::vector<const BitmapValue*>& data, bool& is_first) { res.fastunion(data); is_first = false; // before not set this } ```

…y sql (#52033) (#52999) ### What problem does this PR solve? Problem Summary: cherry-pick from master (#52033) ### Release note None ### Check List (For Author) - Test  - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason  - Behavior changed: - [ ] No. - [ ] Yes.  - Does this need documentation? - [ ] No. - [ ] Yes.  ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

…query sql #52033 (#53000) cherry-pick from master #52033

[Bug](aggregate) fix bitmap_union return error result in query sql
b59f9a1

zhangstar333 mentioned this pull request Jun 20, 2025
[Bug](aggregate) fix bitmap_union return error result in query sql #51866
Merged
16 tasks

HappenLee approved these changes Jun 20, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2025

github-actionsBot added the reviewed label Jun 20, 2025

yiguolei added the dev/2.1.11-merged label Jun 20, 2025

BiteTheDDDDt approved these changes Jun 24, 2025
View reviewed changes

zhangstar333 merged commit ac6e694 into apache:masterJun 24, 2025

morrySnow added dev/3.0.x dev/3.1.x labels Jul 2, 2025

github-actionsBot added dev/3.1.x-conflict dev/3.0.x-conflict labels Jul 2, 2025

dataroaring added the usercase Important user case type label label Jul 9, 2025

This was referenced Jul 9, 2025
[cherry-pick](branch-30) fix bitmap_union return error result in query sql (#52033) #52999
Merged
branch-3.1: [Bug](aggregate) fix bitmap_union return error result in query sql #52033 #53000
Merged

dataroaring added dev/3.0.7-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jul 9, 2025

morrySnow pushed a commit that referenced this pull request Jul 10, 2025
branch-3.1: [Bug](aggregate) fix bitmap_union return error result in …
7db692f
…query sql #52033 (#53000) cherry-pick from master #52033

morrySnow added dev/3.1.0-merged and removed dev/3.1.x dev/3.1.x-conflict labels Jul 10, 2025

yiguolei mentioned this pull request Aug 11, 2025
2.1.11 Release Notes #54550
Closed

gavinchou mentioned this pull request Aug 15, 2025
Release Note 3.0.7 #54868
Closed

Navigation Menu

Uh oh!

[Bug](aggregate) fix bitmap_union return error result in query sql - #52033

[Bug](aggregate) fix bitmap_union return error result in query sql#52033
zhangstar333 merged 1 commit into
apache:masterfrom
zhangstar333:bitmap_union_error

zhangstar333 commented Jun 20, 2025

Uh oh!

hello-stephen commented Jun 20, 2025

Uh oh!

zhangstar333 commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

HappenLee left a comment

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

hello-stephen commented Jun 23, 2025

Uh oh!

hello-stephen commented Jun 23, 2025

Uh oh!

BiteTheDDDDt left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Uh oh!

Conversation

zhangstar333 commented Jun 20, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jun 20, 2025

Uh oh!

zhangstar333 commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

doris-robot commented Jun 20, 2025

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

hello-stephen commented Jun 23, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jun 23, 2025

BE Regression && UT Coverage Report

Uh oh!

BiteTheDDDDt left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants