[Fix](inverted index) skip index compaction for inverted index V2 by airborne12 · Pull Request #38209 · apache/doris

airborne12 · 2024-07-22T08:26:33Z

Proposed changes

Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together.

doris-robot · 2024-07-22T08:26:38Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

airborne12 · 2024-07-22T08:27:04Z

run buildall

github-actions · 2024-07-22T08:33:12Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-22T10:06:24Z

TPC-H: Total hot run time: 40106 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b9b4657afcaa7c2d9082dc9bbf82dcfc7f7a7a5, data reload: false
------ Round 1 ----------------------------------
q1	18138	4699	4303	4303
q2	2024	189	192	189
q3	10527	1228	1152	1152
q4	10214	866	814	814
q5	7565	2714	2708	2708
q6	223	143	140	140
q7	976	598	587	587
q8	9211	2083	2086	2083
q9	8802	6569	6575	6569
q10	8835	3833	3789	3789
q11	468	234	233	233
q12	397	218	226	218
q13	18859	3026	2963	2963
q14	276	239	235	235
q15	521	485	476	476
q16	487	378	379	378
q17	994	635	713	635
q18	8195	7440	7421	7421
q19	5781	1396	1356	1356
q20	658	328	332	328
q21	4910	3245	3244	3244
q22	357	298	285	285
Total cold run time: 118418 ms
Total hot run time: 40106 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4441	4306	4283	4283
q2	396	265	264	264
q3	3083	2921	2908	2908
q4	2004	1760	1758	1758
q5	5538	5602	5471	5471
q6	240	132	131	131
q7	2224	1874	1824	1824
q8	3295	3505	3443	3443
q9	8827	8873	8850	8850
q10	4136	3744	3811	3744
q11	589	494	506	494
q12	804	647	644	644
q13	17000	3141	3197	3141
q14	313	289	301	289
q15	548	487	499	487
q16	478	433	455	433
q17	1824	1513	1510	1510
q18	8145	7968	7892	7892
q19	1777	1463	1564	1463
q20	2182	1863	1911	1863
q21	5073	4792	4754	4754
q22	577	498	494	494
Total cold run time: 73494 ms
Total hot run time: 56140 ms

doris-robot · 2024-07-22T10:17:45Z

TPC-DS: Total hot run time: 175946 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b9b4657afcaa7c2d9082dc9bbf82dcfc7f7a7a5, data reload: false
query1	916	376	373	373
query2	6369	2013	1907	1907
query3	6653	212	224	212
query4	28556	17791	17403	17403
query5	3668	481	493	481
query6	279	175	185	175
query7	4593	308	286	286
query8	256	215	215	215
query9	8604	2513	2468	2468
query10	453	307	286	286
query11	10697	10115	10077	10077
query12	114	98	92	92
query13	1667	382	378	378
query14	10218	8161	7768	7768
query15	223	172	170	170
query16	7666	487	480	480
query17	1648	581	569	569
query18	1946	298	297	297
query19	203	162	164	162
query20	91	90	89	89
query21	213	131	130	130
query22	4620	4310	4277	4277
query23	34325	33505	33721	33505
query24	11107	2965	3006	2965
query25	679	422	451	422
query26	1041	164	158	158
query27	2248	278	287	278
query28	7224	2086	2082	2082
query29	901	674	644	644
query30	270	156	157	156
query31	965	803	765	765
query32	99	54	56	54
query33	768	350	353	350
query34	950	494	495	494
query35	939	809	770	770
query36	1145	974	987	974
query37	153	85	83	83
query38	2919	2817	2775	2775
query39	884	816	806	806
query40	206	124	122	122
query41	51	46	44	44
query42	122	97	103	97
query43	524	494	462	462
query44	1249	731	715	715
query45	198	163	179	163
query46	1095	741	756	741
query47	1867	1820	1807	1807
query48	361	295	290	290
query49	860	443	443	443
query50	791	393	395	393
query51	6844	6662	6625	6625
query52	103	100	97	97
query53	361	294	287	287
query54	913	461	470	461
query55	76	75	76	75
query56	311	276	289	276
query57	1152	1072	1050	1050
query58	281	251	283	251
query59	2957	2738	2680	2680
query60	319	296	300	296
query61	116	115	115	115
query62	806	641	655	641
query63	319	293	288	288
query64	9596	2388	1746	1746
query65	3149	3105	3071	3071
query66	775	349	344	344
query67	15563	15127	15042	15042
query68	4525	527	526	526
query69	717	441	363	363
query70	1203	1083	1075	1075
query71	460	292	280	280
query72	9223	6046	5973	5973
query73	746	327	326	326
query74	6143	5740	5726	5726
query75	3657	2686	2692	2686
query76	3325	958	958	958
query77	675	311	316	311
query78	9813	9064	9055	9055
query79	2955	513	522	513
query80	1830	493	492	492
query81	589	221	228	221
query82	753	145	137	137
query83	355	182	180	180
query84	276	94	99	94
query85	1613	376	367	367
query86	446	311	317	311
query87	3289	3115	3110	3110
query88	4559	2460	2494	2460
query89	499	381	387	381
query90	1884	192	198	192
query91	144	112	112	112
query92	64	52	51	51
query93	4751	496	506	496
query94	1277	291	301	291
query95	409	338	328	328
query96	618	280	274	274
query97	3231	3059	3026	3026
query98	219	207	196	196
query99	1758	1296	1237	1237
Total cold run time: 289241 ms
Total hot run time: 175946 ms

doris-robot · 2024-07-22T10:23:17Z

ClickBench: Total hot run time: 31.06 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b9b4657afcaa7c2d9082dc9bbf82dcfc7f7a7a5, data reload: false
query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.08	0.08
query5	0.50	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.50
query10	0.54	0.54	0.55
query11	0.14	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.76	0.77	0.78
query15	0.85	0.82	0.81
query16	0.37	0.37	0.37
query17	1.01	0.98	1.01
query18	0.22	0.22	0.22
query19	1.77	1.71	1.70
query20	0.01	0.01	0.01
query21	15.39	0.79	0.66
query22	4.26	7.04	2.24
query23	18.30	1.31	1.28
query24	2.07	0.24	0.24
query25	0.14	0.08	0.09
query26	0.30	0.21	0.21
query27	0.46	0.23	0.22
query28	13.21	1.01	1.00
query29	12.61	3.33	3.34
query30	0.25	0.06	0.05
query31	2.88	0.40	0.38
query32	3.28	0.47	0.47
query33	2.89	2.89	2.93
query34	17.19	4.38	4.38
query35	4.38	4.45	4.44
query36	0.66	0.46	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.71 s
Total hot run time: 31.06 s

airborne12 · 2024-07-22T11:21:48Z

run buildall

github-actions · 2024-07-22T11:27:49Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-22T11:59:39Z

TPC-H: Total hot run time: 39907 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6c8bff912eadd098e58c9801897aae852ec53bb, data reload: false
------ Round 1 ----------------------------------
q1	17635	4406	4296	4296
q2	2013	188	189	188
q3	10450	1198	1024	1024
q4	10202	791	790	790
q5	7507	2744	2682	2682
q6	220	138	140	138
q7	958	614	602	602
q8	9219	2046	2087	2046
q9	8872	6591	6611	6591
q10	8791	3738	3773	3738
q11	445	241	245	241
q12	490	232	227	227
q13	18889	2995	2987	2987
q14	281	235	234	234
q15	523	481	472	472
q16	484	384	385	384
q17	978	670	725	670
q18	7909	7390	7404	7390
q19	7175	1439	1373	1373
q20	688	325	325	325
q21	4913	3219	3326	3219
q22	356	290	290	290
Total cold run time: 118998 ms
Total hot run time: 39907 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4450	4325	4316	4316
q2	370	268	264	264
q3	3085	2936	2927	2927
q4	1987	1716	1716	1716
q5	5567	5529	5472	5472
q6	226	136	135	135
q7	2249	1872	1867	1867
q8	3248	3423	3408	3408
q9	8752	8848	8873	8848
q10	4160	3700	3852	3700
q11	599	503	504	503
q12	840	661	645	645
q13	17266	3214	3196	3196
q14	315	277	301	277
q15	545	479	480	479
q16	526	457	437	437
q17	1811	1544	1498	1498
q18	8266	7793	7851	7793
q19	1824	1569	1603	1569
q20	2237	1876	1843	1843
q21	6198	4879	4913	4879
q22	589	509	522	509
Total cold run time: 75110 ms
Total hot run time: 56281 ms

doris-robot · 2024-07-22T12:10:48Z

TPC-DS: Total hot run time: 173865 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6c8bff912eadd098e58c9801897aae852ec53bb, data reload: false
query1	926	374	373	373
query2	6446	1929	1897	1897
query3	6635	211	221	211
query4	28414	17528	17655	17528
query5	3654	463	482	463
query6	265	178	169	169
query7	4603	286	296	286
query8	245	200	192	192
query9	8607	2470	2457	2457
query10	442	295	280	280
query11	10574	10065	10104	10065
query12	117	89	84	84
query13	1650	361	353	353
query14	10192	7666	7618	7618
query15	232	165	165	165
query16	7772	479	427	427
query17	1577	546	531	531
query18	1897	276	282	276
query19	191	151	143	143
query20	87	81	79	79
query21	200	133	128	128
query22	4344	3989	3995	3989
query23	34092	33776	33580	33580
query24	10944	2838	2911	2838
query25	634	386	376	376
query26	1225	153	152	152
query27	2597	275	277	275
query28	7042	2068	2088	2068
query29	891	640	650	640
query30	247	147	154	147
query31	983	768	763	763
query32	97	54	56	54
query33	765	337	349	337
query34	925	488	509	488
query35	905	771	723	723
query36	1135	971	987	971
query37	163	80	83	80
query38	3002	2932	2861	2861
query39	901	853	893	853
query40	203	125	117	117
query41	45	43	43	43
query42	114	98	95	95
query43	521	496	477	477
query44	1119	727	706	706
query45	192	165	160	160
query46	1086	733	741	733
query47	1853	1788	1778	1778
query48	354	290	295	290
query49	845	411	405	405
query50	781	389	393	389
query51	6697	6616	6752	6616
query52	112	93	90	90
query53	357	284	291	284
query54	873	453	445	445
query55	74	75	73	73
query56	281	263	268	263
query57	1122	1046	1054	1046
query58	241	264	253	253
query59	2998	2765	2705	2705
query60	293	270	278	270
query61	98	89	99	89
query62	803	656	648	648
query63	315	286	287	286
query64	9415	2250	1654	1654
query65	3277	3097	3113	3097
query66	747	333	329	329
query67	15488	14989	14934	14934
query68	4451	530	536	530
query69	459	323	367	323
query70	1207	1196	1109	1109
query71	409	281	284	281
query72	7026	5354	5393	5354
query73	759	324	326	324
query74	6245	5728	5631	5631
query75	3379	2731	2679	2679
query76	2407	993	869	869
query77	467	300	306	300
query78	10785	8992	8954	8954
query79	2866	512	519	512
query80	2519	478	459	459
query81	601	219	222	219
query82	977	143	144	143
query83	308	168	168	168
query84	274	91	84	84
query85	2151	312	292	292
query86	479	307	315	307
query87	3344	3139	3081	3081
query88	3674	2443	2444	2443
query89	476	379	382	379
query90	1898	198	193	193
query91	125	101	100	100
query92	63	49	50	49
query93	3247	504	497	497
query94	1274	284	291	284
query95	477	323	321	321
query96	618	291	277	277
query97	3170	3045	3055	3045
query98	230	198	199	198
query99	1583	1265	1300	1265
Total cold run time: 283870 ms
Total hot run time: 173865 ms

doris-robot · 2024-07-22T12:16:21Z

ClickBench: Total hot run time: 30.99 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6c8bff912eadd098e58c9801897aae852ec53bb, data reload: false
query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.06	0.06
query4	1.67	0.09	0.09
query5	0.51	0.50	0.49
query6	1.13	0.72	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.54	0.54	0.53
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.61	0.59	0.57
query14	0.77	0.78	0.78
query15	0.86	0.81	0.82
query16	0.36	0.37	0.35
query17	1.03	0.96	0.97
query18	0.24	0.22	0.22
query19	1.79	1.83	1.74
query20	0.01	0.00	0.00
query21	15.40	0.75	0.67
query22	4.26	6.96	2.04
query23	18.23	1.37	1.37
query24	2.10	0.23	0.22
query25	0.17	0.08	0.09
query26	0.29	0.21	0.22
query27	0.46	0.24	0.24
query28	13.28	1.02	0.99
query29	12.63	3.34	3.36
query30	0.26	0.06	0.06
query31	2.87	0.40	0.39
query32	3.26	0.49	0.47
query33	2.90	2.92	2.92
query34	17.09	4.36	4.36
query35	4.42	4.42	4.40
query36	0.66	0.48	0.47
query37	0.18	0.16	0.16
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.14	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.77 s
Total hot run time: 30.99 s

qidaye

LGTM

github-actions · 2024-07-23T02:18:45Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-23T02:18:47Z

PR approved by anyone and no changes requested.

airborne12 · 2024-07-23T02:23:27Z

run buildall

airborne12 · 2024-07-23T02:24:21Z

run buildall

github-actions · 2024-07-23T02:28:47Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-23T03:36:57Z

TPC-H: Total hot run time: 40122 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bcdd03911a605b876c83756f092a81b1040d0849, data reload: false
------ Round 1 ----------------------------------
q1	17771	4383	4308	4308
q2	2013	195	195	195
q3	10516	1224	1228	1224
q4	10224	830	774	774
q5	7552	2703	2662	2662
q6	224	140	139	139
q7	957	604	605	604
q8	9211	2100	2077	2077
q9	8896	6549	6545	6545
q10	8643	3787	3732	3732
q11	453	237	238	237
q12	395	225	223	223
q13	17781	2978	3003	2978
q14	277	238	242	238
q15	528	491	481	481
q16	511	384	377	377
q17	981	659	781	659
q18	8165	7522	7421	7421
q19	7472	1464	1419	1419
q20	678	313	332	313
q21	4996	3231	3915	3231
q22	343	286	285	285
Total cold run time: 118587 ms
Total hot run time: 40122 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4376	4221	4278	4221
q2	387	268	281	268
q3	3075	2907	3015	2907
q4	1962	1719	1760	1719
q5	5578	5515	5473	5473
q6	229	133	134	133
q7	2269	1825	1867	1825
q8	3275	3438	3435	3435
q9	8762	8883	8692	8692
q10	4148	3757	3852	3757
q11	599	509	505	505
q12	801	678	672	672
q13	16186	3213	3171	3171
q14	341	297	288	288
q15	533	497	476	476
q16	505	446	447	446
q17	1820	1534	1494	1494
q18	8176	7995	7802	7802
q19	1723	1529	1494	1494
q20	2118	1884	1876	1876
q21	5114	5039	4799	4799
q22	588	523	508	508
Total cold run time: 72565 ms
Total hot run time: 55961 ms

doris-robot · 2024-07-23T03:48:16Z

TPC-DS: Total hot run time: 174520 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bcdd03911a605b876c83756f092a81b1040d0849, data reload: false
query1	907	368	367	367
query2	6435	1965	1862	1862
query3	6631	204	225	204
query4	28488	17512	17239	17239
query5	3597	481	493	481
query6	271	184	179	179
query7	4580	295	294	294
query8	257	191	198	191
query9	8488	2436	2411	2411
query10	438	292	282	282
query11	11994	10060	10124	10060
query12	116	87	84	84
query13	1644	374	362	362
query14	10335	7641	7742	7641
query15	234	167	167	167
query16	7780	449	443	443
query17	1554	562	545	545
query18	1853	278	292	278
query19	193	155	154	154
query20	90	80	85	80
query21	202	128	123	123
query22	4357	4105	4171	4105
query23	34194	33718	33738	33718
query24	10945	2918	2936	2918
query25	653	422	423	422
query26	1241	155	161	155
query27	2605	274	294	274
query28	7309	2097	2094	2094
query29	911	648	670	648
query30	246	153	154	153
query31	950	780	755	755
query32	98	54	56	54
query33	743	346	337	337
query34	915	497	520	497
query35	861	778	755	755
query36	1137	983	1004	983
query37	158	80	85	80
query38	2952	2920	2823	2823
query39	885	860	853	853
query40	206	123	118	118
query41	48	45	43	43
query42	118	98	104	98
query43	522	477	461	461
query44	1252	718	732	718
query45	192	164	158	158
query46	1093	755	782	755
query47	1864	1799	1766	1766
query48	380	302	295	295
query49	841	418	423	418
query50	779	396	398	396
query51	6773	6673	6629	6629
query52	104	95	94	94
query53	363	289	289	289
query54	883	448	452	448
query55	77	77	76	76
query56	292	267	289	267
query57	1128	1075	1033	1033
query58	254	247	268	247
query59	2828	2801	2684	2684
query60	300	289	302	289
query61	95	95	91	91
query62	794	658	636	636
query63	326	300	317	300
query64	9359	2218	1695	1695
query65	3175	3119	3136	3119
query66	795	332	332	332
query67	15758	14957	15029	14957
query68	6061	562	554	554
query69	680	446	374	374
query70	1239	1128	1048	1048
query71	503	279	284	279
query72	7820	5526	5611	5526
query73	805	338	327	327
query74	6151	5699	5684	5684
query75	4145	2680	2678	2678
query76	4409	909	924	909
query77	710	309	313	309
query78	10228	8992	9006	8992
query79	3429	534	533	533
query80	1229	487	505	487
query81	582	221	219	219
query82	765	140	139	139
query83	234	169	169	169
query84	283	85	91	85
query85	1375	310	313	310
query86	448	313	282	282
query87	3277	3114	3119	3114
query88	4174	2384	2502	2384
query89	493	383	397	383
query90	1871	196	200	196
query91	134	100	102	100
query92	65	50	48	48
query93	4446	533	512	512
query94	876	297	301	297
query95	411	319	325	319
query96	606	279	281	279
query97	3205	3061	3022	3022
query98	229	207	201	201
query99	1519	1281	1278	1278
Total cold run time: 290248 ms
Total hot run time: 174520 ms

doris-robot · 2024-07-23T03:53:48Z

ClickBench: Total hot run time: 30.34 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bcdd03911a605b876c83756f092a81b1040d0849, data reload: false
query1	0.05	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.51	0.48	0.49
query6	1.12	0.71	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.50	0.48
query10	0.52	0.54	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.13
query13	0.61	0.59	0.60
query14	0.76	0.78	0.79
query15	0.85	0.81	0.82
query16	0.37	0.36	0.36
query17	0.98	0.96	0.99
query18	0.24	0.22	0.21
query19	1.77	1.72	1.67
query20	0.01	0.02	0.01
query21	15.41	0.76	0.67
query22	3.84	8.02	1.67
query23	18.24	1.42	1.28
query24	2.16	0.23	0.22
query25	0.15	0.08	0.08
query26	0.30	0.21	0.22
query27	0.45	0.23	0.23
query28	13.25	1.01	1.00
query29	12.65	3.27	3.31
query30	0.25	0.06	0.05
query31	2.87	0.40	0.39
query32	3.25	0.47	0.47
query33	2.90	2.93	2.98
query34	17.02	4.34	4.34
query35	4.39	4.40	4.40
query36	0.66	0.47	0.45
query37	0.19	0.16	0.17
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.14	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.2 s
Total hot run time: 30.34 s

airborne12 · 2024-07-23T06:00:02Z

run buildall

github-actions · 2024-07-23T06:06:15Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-23T06:47:10Z

TPC-H: Total hot run time: 39712 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d322c54d72f00ee882c72e00d6905d5a8b1d6681, data reload: false
------ Round 1 ----------------------------------
q1	17598	4331	4276	4276
q2	2008	188	181	181
q3	10458	1240	1170	1170
q4	10191	840	887	840
q5	7547	2692	2732	2692
q6	219	136	136	136
q7	952	595	581	581
q8	9220	2055	2076	2055
q9	8687	6570	6509	6509
q10	8790	3777	3730	3730
q11	477	235	242	235
q12	436	220	219	219
q13	17765	2960	2975	2960
q14	288	241	249	241
q15	516	482	498	482
q16	505	403	379	379
q17	961	647	696	647
q18	7983	7386	7378	7378
q19	7715	1433	1383	1383
q20	681	328	312	312
q21	4931	3028	3168	3028
q22	335	287	278	278
Total cold run time: 118263 ms
Total hot run time: 39712 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4406	4256	4231	4231
q2	384	278	257	257
q3	2975	2913	2933	2913
q4	1980	1633	1763	1633
q5	5606	5502	5452	5452
q6	222	140	136	136
q7	2177	1906	1857	1857
q8	3223	3411	3448	3411
q9	8722	8731	8807	8731
q10	4096	3779	3842	3779
q11	611	495	500	495
q12	784	627	645	627
q13	17279	3163	3159	3159
q14	330	293	293	293
q15	533	481	491	481
q16	528	441	430	430
q17	1787	1504	1523	1504
q18	8142	7871	7889	7871
q19	1767	1469	1492	1469
q20	2980	1849	1860	1849
q21	9459	4670	4696	4670
q22	594	506	495	495
Total cold run time: 78585 ms
Total hot run time: 55743 ms

doris-robot · 2024-07-23T06:58:33Z

TPC-DS: Total hot run time: 174085 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d322c54d72f00ee882c72e00d6905d5a8b1d6681, data reload: false
query1	898	373	362	362
query2	6415	1935	1835	1835
query3	6649	201	210	201
query4	27598	17576	17148	17148
query5	3765	466	490	466
query6	261	172	159	159
query7	4580	294	282	282
query8	235	190	190	190
query9	8448	2438	2397	2397
query10	440	275	285	275
query11	10657	9964	10003	9964
query12	112	84	81	81
query13	1639	391	361	361
query14	9566	7618	7631	7618
query15	219	167	167	167
query16	7739	474	487	474
query17	1149	552	523	523
query18	1941	281	294	281
query19	192	147	147	147
query20	93	81	82	81
query21	211	127	123	123
query22	4237	4013	3943	3943
query23	33891	33611	33668	33611
query24	11255	3002	2935	2935
query25	640	379	378	378
query26	907	149	147	147
query27	2379	279	288	279
query28	6721	2083	2097	2083
query29	922	639	637	637
query30	251	152	149	149
query31	992	762	813	762
query32	93	51	54	51
query33	749	335	323	323
query34	946	480	493	480
query35	868	755	747	747
query36	1164	973	965	965
query37	141	81	83	81
query38	2970	2945	2822	2822
query39	881	862	818	818
query40	202	118	117	117
query41	49	44	43	43
query42	122	97	110	97
query43	507	481	474	474
query44	1245	741	741	741
query45	194	167	163	163
query46	1122	743	732	732
query47	1857	1774	1785	1774
query48	376	285	289	285
query49	836	399	407	399
query50	767	384	393	384
query51	6825	6623	6608	6608
query52	105	96	94	94
query53	358	288	292	288
query54	874	446	450	446
query55	71	77	74	74
query56	284	281	270	270
query57	1111	1086	1043	1043
query58	244	272	266	266
query59	2826	2627	2611	2611
query60	304	274	276	274
query61	110	115	116	115
query62	803	658	660	658
query63	330	291	294	291
query64	9342	2295	7526	2295
query65	3142	3082	3097	3082
query66	764	338	340	338
query67	15476	15059	15233	15059
query68	4548	556	568	556
query69	620	439	380	380
query70	1082	1146	1115	1115
query71	415	289	295	289
query72	7209	5567	5254	5254
query73	745	329	323	323
query74	6116	5688	5699	5688
query75	3443	2677	2679	2677
query76	2785	945	915	915
query77	674	318	313	313
query78	9600	8997	9028	8997
query79	8278	536	537	536
query80	2444	495	500	495
query81	600	222	230	222
query82	1528	261	131	131
query83	315	160	161	160
query84	280	87	87	87
query85	1756	309	343	309
query86	481	344	317	317
query87	3307	3091	3063	3063
query88	5148	2438	2380	2380
query89	491	376	390	376
query90	1832	184	187	184
query91	128	98	99	98
query92	61	50	47	47
query93	5985	516	521	516
query94	1191	273	281	273
query95	401	310	312	310
query96	613	283	278	278
query97	3250	2961	3001	2961
query98	217	202	200	200
query99	1649	1286	1268	1268
Total cold run time: 290252 ms
Total hot run time: 174085 ms

doris-robot · 2024-07-23T07:04:04Z

ClickBench: Total hot run time: 30.77 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d322c54d72f00ee882c72e00d6905d5a8b1d6681, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.67	0.07	0.07
query5	0.50	0.49	0.47
query6	1.12	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.55	0.49	0.49
query10	0.53	0.54	0.53
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.61	0.59	0.58
query14	0.75	0.78	0.77
query15	0.84	0.80	0.82
query16	0.36	0.35	0.37
query17	1.04	1.02	0.98
query18	0.22	0.21	0.22
query19	1.81	1.67	1.71
query20	0.01	0.01	0.01
query21	15.41	0.78	0.65
query22	4.15	6.96	2.18
query23	18.32	1.37	1.26
query24	2.16	0.25	0.22
query25	0.15	0.09	0.08
query26	0.28	0.22	0.21
query27	0.46	0.23	0.23
query28	13.20	1.00	0.99
query29	12.64	3.32	3.29
query30	0.26	0.06	0.05
query31	2.88	0.38	0.38
query32	3.31	0.46	0.47
query33	2.95	2.91	2.98
query34	17.08	4.33	4.31
query35	4.36	4.41	4.41
query36	0.65	0.46	0.48
query37	0.19	0.16	0.16
query38	0.16	0.16	0.14
query39	0.04	0.03	0.04
query40	0.14	0.13	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.74 s
Total hot run time: 30.77 s

csun5285

LGTM

csun5285

LGTM

github-actions · 2024-07-23T08:12:32Z

PR approved by at least one committer and no changes requested.

zzzxl1993

LGTM

…ache#38209) ## Proposed changes Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together.

…ache#38209) Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together.

#38273) ## Proposed changes Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together. backport #38209

#38278) Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together. backport #38209

…ache#38209) ## Proposed changes Enabling index compaction in the inverted index V2 format currently causes unexpected errors, especially in the case of tables with hybrid indexes, such as BKD index and Fulltext index together.

airborne12 force-pushed the cloud-inverted-index branch from 5b9b465 to d6c8bffCompare July 22, 2024 11:21

[Fix](inverted index) skip index compaction for inverted index V2
70d2df7

qidaye previously approved these changes Jul 23, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024

github-actionsBot added the reviewed label Jul 23, 2024

airborne12 dismissed qidaye’s stale review via bcdd039July 23, 2024 02:22

airborne12 force-pushed the cloud-inverted-index branch from d6c8bff to bcdd039Compare July 23, 2024 02:22

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jul 23, 2024

disable test_index_compaction_failure_injection
d322c54

airborne12 force-pushed the cloud-inverted-index branch from bcdd039 to d322c54Compare July 23, 2024 05:59

csun5285 reviewed Jul 23, 2024
View reviewed changes

csun5285 approved these changes Jul 23, 2024
View reviewed changes

qidaye approved these changes Jul 23, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024

zzzxl1993 approved these changes Jul 23, 2024
View reviewed changes

airborne12 merged commit 4f4d4b0 into apache:masterJul 23, 2024

airborne12 deleted the cloud-inverted-index branch July 23, 2024 08:33

airborne12 added dev/2.1.x dev/3.0.x labels Jul 24, 2024

airborne12 mentioned this pull request Jul 24, 2024
[Pick 3.0](inverted index) skip index compaction for inverted index V2 #38273
Merged

airborne12 mentioned this pull request Jul 24, 2024
[Pick 2.1](inverted index) skip index compaction for inverted index V2 #38278
Merged

airborne12 added dev/3.0.1-merged and removed dev/3.0.x labels Jul 24, 2024

airborne12 added dev/2.1.6-merged and removed dev/2.1.x labels Jul 24, 2024

Uh oh!

Conversation

airborne12 commented Jul 22, 2024

Proposed changes

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

airborne12 commented Jul 22, 2024

Uh oh!

github-actionsBot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

airborne12 commented Jul 22, 2024

Uh oh!

github-actionsBot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

qidaye left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 23, 2024

Uh oh!

github-actionsBot commented Jul 23, 2024

Uh oh!

airborne12 commented Jul 23, 2024

Uh oh!

airborne12 commented Jul 23, 2024

Uh oh!

github-actionsBot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

airborne12 commented Jul 23, 2024

Uh oh!

github-actionsBot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

doris-robot commented Jul 23, 2024

Uh oh!

csun5285 left a comment

Choose a reason for hiding this comment

Uh oh!

csun5285 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 23, 2024

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants