[fix](bloom filter)Fix rename column with bloom filter by qidaye · Pull Request #38431 · apache/doris

qidaye · 2024-07-26T18:07:40Z

Proposed changes

Issue Number: close#38643

doris-robot · 2024-07-26T18:07:45Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

qidaye · 2024-07-26T18:07:48Z

run buildall

doris-robot · 2024-07-26T18:32:23Z

TPC-H: Total hot run time: 39416 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c26e507fd4c88f59b5679f5ed712e612a6be77b, data reload: false
------ Round 1 ----------------------------------
q1	18636	4502	4405	4405
q2	2529	201	198	198
q3	11378	1233	1053	1053
q4	10563	711	714	711
q5	7601	2829	2696	2696
q6	226	144	138	138
q7	966	612	632	612
q8	9359	1905	1876	1876
q9	8630	6507	6624	6507
q10	8589	3853	3800	3800
q11	459	251	256	251
q12	400	218	219	218
q13	18793	2963	2979	2963
q14	281	242	232	232
q15	516	483	497	483
q16	478	394	390	390
q17	954	633	711	633
q18	8058	7407	7368	7368
q19	4077	1091	1075	1075
q20	694	320	342	320
q21	5165	3195	3376	3195
q22	364	301	292	292
Total cold run time: 118716 ms
Total hot run time: 39416 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4418	4283	4237	4237
q2	364	274	280	274
q3	2978	2745	2734	2734
q4	1905	1590	1583	1583
q5	5265	5293	5317	5293
q6	221	129	129	129
q7	2145	1749	1747	1747
q8	3174	3352	3275	3275
q9	8367	8290	8292	8290
q10	3954	3697	3754	3697
q11	594	477	481	477
q12	780	581	590	581
q13	17447	2975	3011	2975
q14	295	291	276	276
q15	507	467	470	467
q16	461	411	411	411
q17	1744	1481	1476	1476
q18	7570	7471	7323	7323
q19	2297	1564	1508	1508
q20	2005	1807	1745	1745
q21	4806	4699	4775	4699
q22	587	511	520	511
Total cold run time: 71884 ms
Total hot run time: 53708 ms

doris-robot · 2024-07-26T18:44:01Z

TPC-DS: Total hot run time: 173122 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c26e507fd4c88f59b5679f5ed712e612a6be77b, data reload: false
query1	890	372	362	362
query2	6441	1918	1859	1859
query3	6665	208	213	208
query4	28493	17650	17391	17391
query5	4227	487	487	487
query6	263	164	157	157
query7	4581	306	295	295
query8	240	201	208	201
query9	8592	2440	2428	2428
query10	452	291	272	272
query11	11186	10132	10159	10132
query12	138	91	84	84
query13	1618	367	391	367
query14	10224	7695	7579	7579
query15	222	171	163	163
query16	7762	497	503	497
query17	1614	605	564	564
query18	1980	300	303	300
query19	202	147	150	147
query20	96	95	88	88
query21	212	105	102	102
query22	4432	3984	4202	3984
query23	33864	33178	33155	33155
query24	12157	2919	2834	2834
query25	699	395	401	395
query26	1774	153	157	153
query27	2927	272	277	272
query28	7374	2013	1988	1988
query29	1149	442	427	427
query30	289	149	149	149
query31	977	753	745	745
query32	96	53	55	53
query33	818	356	338	338
query34	922	470	469	469
query35	860	788	741	741
query36	1132	924	913	913
query37	290	88	87	87
query38	2880	2818	2796	2796
query39	861	817	801	801
query40	273	121	118	118
query41	50	48	49	48
query42	123	101	99	99
query43	509	489	455	455
query44	1203	720	715	715
query45	209	183	183	183
query46	1156	712	736	712
query47	1914	1795	1794	1794
query48	373	303	300	300
query49	1206	435	426	426
query50	860	412	405	405
query51	6821	6762	6677	6677
query52	100	94	91	91
query53	265	198	191	191
query54	1003	447	453	447
query55	78	78	77	77
query56	312	288	286	286
query57	1186	1060	1139	1060
query58	268	271	283	271
query59	2905	2703	2580	2580
query60	314	281	290	281
query61	100	97	92	92
query62	853	633	647	633
query63	218	185	184	184
query64	10575	2316	1662	1662
query65	3196	3103	3107	3103
query66	1380	330	325	325
query67	15755	14888	14757	14757
query68	8875	562	571	562
query69	747	413	361	361
query70	1345	1124	1117	1117
query71	533	268	272	268
query72	8789	5677	5300	5300
query73	2230	336	327	327
query74	6103	5732	5719	5719
query75	5351	2665	2645	2645
query76	5405	1057	968	968
query77	779	312	309	309
query78	11549	9336	10813	9336
query79	10383	515	519	515
query80	1492	550	522	522
query81	596	223	216	216
query82	311	137	137	137
query83	356	184	175	175
query84	275	86	80	80
query85	768	318	293	293
query86	357	307	327	307
query87	3445	3080	3110	3080
query88	4629	2465	2530	2465
query89	415	283	284	283
query90	2388	200	201	200
query91	126	103	103	103
query92	60	50	52	50
query93	2919	540	524	524
query94	1159	313	288	288
query95	362	268	264	264
query96	650	279	281	279
query97	3274	2996	3036	2996
query98	227	205	204	204
query99	1676	1278	1274	1274
Total cold run time: 310378 ms
Total hot run time: 173122 ms

doris-robot · 2024-07-26T18:49:33Z

ClickBench: Total hot run time: 30.13 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c26e507fd4c88f59b5679f5ed712e612a6be77b, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.49	0.50	0.50
query6	1.14	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.48
query10	0.54	0.53	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.13
query13	0.60	0.58	0.58
query14	0.76	0.78	0.78
query15	0.85	0.81	0.81
query16	0.36	0.35	0.35
query17	1.02	1.03	1.03
query18	0.23	0.22	0.21
query19	1.85	1.82	1.78
query20	0.01	0.01	0.01
query21	15.39	0.78	0.66
query22	4.52	7.53	1.42
query23	18.30	1.41	1.28
query24	2.13	0.24	0.21
query25	0.16	0.09	0.09
query26	0.30	0.22	0.21
query27	0.45	0.23	0.23
query28	13.24	1.04	1.01
query29	12.62	3.29	3.22
query30	0.25	0.06	0.05
query31	2.87	0.38	0.39
query32	3.28	0.47	0.48
query33	2.88	2.92	2.86
query34	17.33	4.40	4.39
query35	4.48	4.38	4.44
query36	0.64	0.46	0.46
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.05	0.03	0.04
query40	0.16	0.12	0.13
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 110.36 s
Total hot run time: 30.13 s

xiaokang

LGTM

github-actions · 2024-07-28T11:28:58Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-28T11:29:00Z

PR approved by anyone and no changes requested.

qidaye · 2024-07-30T03:37:58Z

run buildall

doris-robot · 2024-07-30T04:13:31Z

TPC-H: Total hot run time: 41631 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ba7716a05e7823cd4ab686d61c7e695ef845b21a, data reload: false
------ Round 1 ----------------------------------
q1	17598	4068	4018	4018
q2	2029	206	200	200
q3	10441	1290	1368	1290
q4	10163	821	946	821
q5	7681	2982	2945	2945
q6	223	143	135	135
q7	1036	611	612	611
q8	9443	1882	1938	1882
q9	8455	6590	6573	6573
q10	8734	3840	3820	3820
q11	435	258	250	250
q12	418	234	237	234
q13	17755	2959	2951	2951
q14	278	244	242	242
q15	530	484	487	484
q16	526	402	400	400
q17	957	903	909	903
q18	7959	7364	7248	7248
q19	1547	1217	1207	1207
q20	571	332	364	332
q21	5290	4797	4826	4797
q22	354	288	292	288
Total cold run time: 112423 ms
Total hot run time: 41631 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4020	4030	4006	4006
q2	330	219	226	219
q3	3004	3034	3149	3034
q4	2013	2041	1977	1977
q5	5573	5529	5447	5447
q6	221	132	131	131
q7	2134	1847	1836	1836
q8	3334	3376	3376	3376
q9	8676	8695	8867	8695
q10	3938	4049	3890	3890
q11	565	472	463	463
q12	780	614	616	614
q13	15549	3140	3167	3140
q14	317	254	282	254
q15	537	475	486	475
q16	471	413	432	413
q17	1747	1733	1756	1733
q18	8129	7698	7781	7698
q19	2534	1745	1739	1739
q20	2106	1824	1845	1824
q21	5678	5573	5385	5385
q22	557	499	481	481
Total cold run time: 72213 ms
Total hot run time: 56830 ms

doris-robot · 2024-07-30T04:24:17Z

TPC-DS: Total hot run time: 171686 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ba7716a05e7823cd4ab686d61c7e695ef845b21a, data reload: false
query1	923	372	374	372
query2	6458	1735	1741	1735
query3	6677	225	222	222
query4	20004	17197	17262	17197
query5	3662	519	528	519
query6	300	170	183	170
query7	4606	304	297	297
query8	256	198	195	195
query9	8537	2405	2390	2390
query10	437	283	274	274
query11	10407	10145	9915	9915
query12	123	90	87	87
query13	1633	387	380	380
query14	9721	8548	8576	8548
query15	217	167	169	167
query16	6969	470	435	435
query17	958	590	560	560
query18	1914	294	292	292
query19	198	150	152	150
query20	96	87	90	87
query21	209	108	106	106
query22	4088	3997	3894	3894
query23	34230	33735	33662	33662
query24	9341	3192	3076	3076
query25	677	424	426	424
query26	1398	158	161	158
query27	2997	295	293	293
query28	7359	2028	2016	2016
query29	1021	443	446	443
query30	239	152	156	152
query31	951	785	780	780
query32	104	57	60	57
query33	676	330	344	330
query34	905	522	549	522
query35	905	764	766	764
query36	1066	901	866	866
query37	194	83	80	80
query38	2967	2852	2761	2761
query39	874	837	836	836
query40	250	113	110	110
query41	49	43	43	43
query42	117	97	107	97
query43	467	423	420	420
query44	1178	725	734	725
query45	209	184	183	183
query46	1114	838	774	774
query47	1781	1689	1710	1689
query48	384	305	291	291
query49	852	436	428	428
query50	926	444	447	444
query51	6773	6731	6651	6651
query52	101	88	108	88
query53	253	179	181	179
query54	641	460	463	460
query55	76	74	73	73
query56	279	258	259	258
query57	1117	1024	1018	1018
query58	269	284	280	280
query59	2557	2587	2525	2525
query60	302	270	273	270
query61	102	115	97	97
query62	894	652	640	640
query63	218	186	186	186
query64	5843	1944	1892	1892
query65	3175	3110	3126	3110
query66	1108	328	341	328
query67	15447	14699	14815	14699
query68	5325	576	591	576
query69	681	411	306	306
query70	1148	1075	1076	1075
query71	455	289	279	279
query72	7653	2705	2507	2507
query73	930	331	333	331
query74	6015	5547	5594	5547
query75	3655	2746	2763	2746
query76	3161	1420	1425	1420
query77	583	324	318	318
query78	9459	8912	8978	8912
query79	1926	538	551	538
query80	834	512	504	504
query81	564	229	227	227
query82	772	136	130	130
query83	195	174	169	169
query84	268	80	83	80
query85	1373	334	323	323
query86	488	306	305	305
query87	3297	3091	3121	3091
query88	3761	2494	2499	2494
query89	404	298	294	294
query90	1757	199	199	199
query91	131	104	106	104
query92	60	52	51	51
query93	1614	625	623	623
query94	756	293	304	293
query95	391	277	280	277
query96	607	281	282	281
query97	3242	3041	3053	3041
query98	232	204	196	196
query99	1582	1309	1279	1279
Total cold run time: 264098 ms
Total hot run time: 171686 ms

doris-robot · 2024-07-30T04:29:46Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ba7716a05e7823cd4ab686d61c7e695ef845b21a, data reload: false
query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.06	0.06
query5	0.48	0.50	0.48
query6	1.13	0.71	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.57	0.53	0.51
query10	0.57	0.56	0.57
query11	0.15	0.12	0.12
query12	0.16	0.13	0.13
query13	0.61	0.61	0.60
query14	0.78	0.78	0.81
query15	0.90	0.86	0.87
query16	0.35	0.37	0.36
query17	0.99	1.03	1.01
query18	0.23	0.22	0.21
query19	1.82	1.72	1.76
query20	0.02	0.01	0.01
query21	15.42	0.81	0.67
query22	3.97	6.77	1.64
query23	17.96	1.40	1.37
query24	2.26	0.23	0.22
query25	0.20	0.09	0.09
query26	0.32	0.22	0.22
query27	0.46	0.24	0.24
query28	13.18	1.03	0.97
query29	12.59	3.30	3.30
query30	0.25	0.06	0.06
query31	2.86	0.42	0.42
query32	3.23	0.51	0.48
query33	2.99	2.95	2.96
query34	15.41	4.28	4.26
query35	4.31	4.29	4.31
query36	0.68	0.48	0.49
query37	0.20	0.17	0.16
query38	0.18	0.16	0.15
query39	0.04	0.04	0.03
query40	0.17	0.13	0.14
query41	0.10	0.06	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.72 s
Total hot run time: 30.55 s

airborne12 · 2024-07-30T13:13:46Z

need to fill Proposed changes

xiaokang

LGTM

github-actions · 2024-07-30T13:24:55Z

PR approved by at least one committer and no changes requested.

qidaye · 2024-07-31T16:53:07Z

need to fill Proposed changes

Added in issue.

airborne12

LGTM

Issue Number: closeapache#38643

backport #38431

Issue Number: closeapache#38643

Issue Number: close#38643

Issue Number: closeapache#38643

[fix](bloom filter)Fix rename column with bloom filter
7c26e50

qidaye requested review from CalvinKirs, dataroaring and morningman as code owners July 26, 2024 18:07

xiaokang previously approved these changes Jul 28, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 28, 2024

xiaokang reviewed Jul 28, 2024
View reviewed changes

Comment threadfe/fe-core/src/main/java/org/apache/doris/catalog/Env.java Outdated

fix
ba7716a

qidaye dismissed xiaokang’s stale review via ba7716aJuly 30, 2024 03:37

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jul 30, 2024

xiaokang approved these changes Jul 30, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 30, 2024

xiaokang added 2.1.0-conflict dev/2.1.x doing labels Jul 31, 2024

airborne12 approved these changes Aug 1, 2024
View reviewed changes

hello-stephen approved these changes Aug 1, 2024
View reviewed changes

CalvinKirs approved these changes Aug 1, 2024
View reviewed changes

qidaye merged commit 1eff4e1 into apache:masterAug 1, 2024

qidaye deleted the fix_rename branch August 1, 2024 02:40

qidaye added a commit to qidaye/incubator-doris that referenced this pull request Aug 1, 2024
[fix](bloom filter)Fix rename column with bloom filter (apache#38431)
b81f42a
Issue Number: closeapache#38643

qidaye mentioned this pull request Aug 1, 2024
[fix](bloom filter)Fix rename column with bloom filter (#38431) #38662
Merged

qidaye removed the 2.1.0-conflict label Aug 1, 2024

yiguolei pushed a commit that referenced this pull request Aug 1, 2024
[fix](bloom filter)Fix rename column with bloom filter (#38431) (#38662)
555dccb
backport #38431

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Aug 1, 2024

qidaye removed the doing label Aug 7, 2024

feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
[fix](bloom filter)Fix rename column with bloom filter (apache#38431)
fb2d6cd
Issue Number: closeapache#38643

dataroaring added the dev/3.0.2-merged label Aug 11, 2024

dataroaring pushed a commit that referenced this pull request Aug 11, 2024
[fix](bloom filter)Fix rename column with bloom filter (#38431)
d484c65
Issue Number: close#38643

dataroaring pushed a commit that referenced this pull request Aug 16, 2024
[fix](bloom filter)Fix rename column with bloom filter (#38431)
9d03dd6
Issue Number: close#38643

gavinchou mentioned this pull request Oct 13, 2024
Release Note 3.0.2 #41558
Closed

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
[fix](bloom filter)Fix rename column with bloom filter (apache#38431)
a348f69
Issue Number: closeapache#38643

Uh oh!

Conversation

qidaye commented Jul 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Jul 26, 2024

Uh oh!

qidaye commented Jul 26, 2024

Uh oh!

doris-robot commented Jul 26, 2024

Uh oh!

doris-robot commented Jul 26, 2024

Uh oh!

doris-robot commented Jul 26, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 28, 2024

Uh oh!

github-actionsBot commented Jul 28, 2024

Uh oh!

Uh oh!

qidaye commented Jul 30, 2024

Uh oh!

doris-robot commented Jul 30, 2024

Uh oh!

doris-robot commented Jul 30, 2024

Uh oh!

doris-robot commented Jul 30, 2024

Uh oh!

airborne12 commented Jul 30, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 30, 2024

Uh oh!

qidaye commented Jul 31, 2024

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

qidaye commented Jul 26, 2024•
edited
Loading