[Fix] (schema change) disable convert light schema change (#28205) by luwei16 · Pull Request #29300 · apache/doris

luwei16 · 2023-12-29T08:12:30Z

branch-2.0 #28205

luwei16 · 2023-12-29T08:12:47Z

run buildall

dataroaring

LGTM

github-actions · 2023-12-29T08:22:07Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-12-29T08:22:09Z

PR approved by anyone and no changes requested.

doris-robot · 2023-12-29T09:15:28Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 9cf43434f658b218761a363764b1cc77ce48ef6a, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5468	5201	5060	5060
q2	401	172	178	172
q3	1458	1165	1182	1165
q4	1093	825	793	793
q5	3107	3097	3124	3097
q6	237	144	142	142
q7	989	561	541	541
q8	2163	2294	2195	2195
q9	6728	6664	6712	6664
q10	3183	3166	3175	3166
q11	352	224	243	224
q12	396	247	242	242
q13	4418	3666	3617	3617
q14	251	224	223	223
q15	610	580	576	576
q16	465	421	405	405
q17	1044	494	562	494
q18	7132	6844	8050	6844
q19	1646	1539	1638	1539
q20	583	381	359	359
q21	2990	2495	2419	2419
q22	399	338	331	331
Total cold run time: 45113 ms
Total hot run time: 40268 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5153	5064	5135	5064
q2	333	250	245	245
q3	3391	3326	3316	3316
q4	2173	2021	2021	2021
q5	5958	5938	5925	5925
q6	239	133	137	133
q7	2406	1960	1955	1955
q8	3549	3667	3671	3667
q9	9096	9060	9041	9041
q10	3883	3940	3941	3940
q11	593	484	508	484
q12	796	658	636	636
q13	3902	3214	3177	3177
q14	294	289	277	277
q15	606	555	544	544
q16	575	517	548	517
q17	2020	1814	1789	1789
q18	8889	8421	8461	8421
q19	1774	1702	1772	1702
q20	2276	1995	1986	1986
q21	5744	5438	5351	5351
q22	581	458	525	458
Total cold run time: 64231 ms
Total hot run time: 60649 ms

doris-robot · 2023-12-29T09:28:23Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.66 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.3 seconds inserted 10000000 Rows, about 353K ops/s
storage size: 17183991365 Bytes

yiguolei · 2023-12-30T12:20:10Z

run buildall

doris-robot · 2023-12-30T12:39:39Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit a3a01a003e3cbfe29c1a2e952a075126ebff4337, data reload: false
------ Round 1 ----------------------------------
q1	18462	5191	5151	5151
q2	2034	163	151	151
q3	10597	1123	1111	1111
q4	10197	788	795	788
q5	7816	2989	3034	2989
q6	213	134	138	134
q7	945	515	551	515
q8	9344	2064	2039	2039
q9	6884	6446	6428	6428
q10	8258	3049	3052	3049
q11	428	234	220	220
q12	390	254	233	233
q13	18000	3641	3601	3601
q14	240	206	212	206
q15	581	542	537	537
q16	470	410	395	395
q17	966	489	478	478
q18	7386	6702	6740	6702
q19	1604	1401	1385	1385
q20	747	340	340	340
q21	2803	2345	2333	2333
q22	380	322	322	322
Total cold run time: 108745 ms
Total hot run time: 39107 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5195	5096	5099	5096
q2	337	270	259	259
q3	3327	3288	3256	3256
q4	2105	2026	1975	1975
q5	5811	5821	5777	5777
q6	216	130	129	129
q7	2316	1902	1893	1893
q8	3372	3435	3471	3435
q9	8829	8805	8821	8805
q10	3800	3844	3858	3844
q11	591	488	474	474
q12	818	635	657	635
q13	8908	3211	3182	3182
q14	285	273	271	271
q15	603	545	536	536
q16	555	497	539	497
q17	1954	1751	1747	1747
q18	8790	8380	8251	8251
q19	1643	1592	1617	1592
q20	2213	1982	1971	1971
q21	5597	5264	5354	5264
q22	544	465	495	465
Total cold run time: 67809 ms
Total hot run time: 59354 ms

doris-robot · 2023-12-30T12:40:11Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit a3a01a003e3cbfe29c1a2e952a075126ebff4337, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5433	5165	5169	5165
q2	396	169	159	159
q3	1474	1240	1209	1209
q4	1100	823	803	803
q5	3159	3114	3125	3114
q6	221	130	132	130
q7	989	551	524	524
q8	2166	2226	2226	2226
q9	6704	6654	6666	6654
q10	3171	3196	3146	3146
q11	357	218	230	218
q12	380	231	231	231
q13	4406	3686	3661	3661
q14	244	206	217	206
q15	601	558	551	551
q16	476	409	407	407
q17	1054	565	553	553
q18	7039	6723	7224	6723
q19	1624	1512	1462	1462
q20	607	363	347	347
q21	2882	2424	2514	2424
q22	392	334	327	327
Total cold run time: 44875 ms
Total hot run time: 40240 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5173	5040	5108	5040
q2	333	256	243	243
q3	3337	3325	3330	3325
q4	2141	2020	2020	2020
q5	5939	5929	5973	5929
q6	232	127	126	126
q7	2393	1950	1926	1926
q8	3568	3677	3662	3662
q9	9050	9011	9019	9011
q10	3887	3907	3929	3907
q11	591	485	479	479
q12	807	674	619	619
q13	3906	3222	3214	3214
q14	286	309	282	282
q15	618	535	552	535
q16	541	543	506	506
q17	2016	1844	1809	1809
q18	8761	8319	8391	8319
q19	1733	1703	1701	1701
q20	2309	1996	1987	1987
q21	5775	5318	5266	5266
q22	580	508	507	507
Total cold run time: 63976 ms
Total hot run time: 60413 ms

doris-robot · 2023-12-30T12:51:12Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit a3a01a003e3cbfe29c1a2e952a075126ebff4337, data reload: false
run tpcds-sf100 query with default conf and session variables
query1	899	370	346	346
query2	5445	1979	2031	1979
query3	5721	207	204	204
query4	26014	22535	22355	22355
query5	5252	570	566	566
query6	265	185	183	183
query7	4375	282	276	276
query8	221	205	203	203
query9	7724	2678	2607	2607
query10	428	280	256	256
query11	16172	15451	15650	15451
query12	145	85	79	79
query13	1551	348	330	330
query14	11725	7207	7255	7207
query15	233	193	192	192
query16	6205	295	277	277
query17	1592	501	502	501
query18	1837	273	261	261
query19	262	144	142	142
query20	83	80	81	80
query21	174	99	93	93
query22	5027	4883	4476	4476
query23	32081	31113	31212	31113
query24	11907	2831	2783	2783
query25	604	348	352	348
query26	1716	141	150	141
query27	2780	276	279	276
query28	6826	2008	1998	1998
query29	1860	404	387	387
query30	283	146	150	146
query31	981	761	774	761
query32	86	62	64	62
query33	724	278	279	278
query34	835	455	452	452
query35	907	784	774	774
query36	1322	1248	1249	1248
query37	174	80	79	79
query38	3382	3311	3276	3276
query39	1328	1317	1280	1280
query40	294	98	93	93
query41	41	35	35	35
query42	98	91	98	91
query43	525	483	505	483
query44	1128	707	740	707
query45	202	193	191	191
query46	1084	637	635	635
query47	1661	1626	1523	1523
query48	353	271	272	271
query49	1114	334	337	334
query50	766	341	347	341
query51	5353	5321	5196	5196
query52	96	93	93	93
query53	219	150	158	150
query54	1145	570	611	570
query55	106	90	85	85
query56	219	203	206	203
query57	1002	923	975	923
query58	222	208	219	208
query59	2864	2598	2739	2598
query60	265	233	237	233
query61	84	82	81	81
query62	643	451	466	451
query63	162	161	152	152
query64	5438	1712	1648	1648
query65	3355	3286	3261	3261
query66	1285	351	352	351
query67	15547	15626	15064	15064
query68	12757	516	543	516
query69	526	259	259	259
query70	1694	1556	1427	1427
query71	482	237	229	229
query72	5645	3548	3530	3530
query73	2964	319	315	315
query74	7123	6478	6490	6478
query75	5333	2272	2281	2272
query76	6221	1116	1176	1116
query77	652	287	298	287
query78	9132	8911	8647	8647
query79	4336	504	515	504
query80	661	370	371	370
query81	454	210	206	206
query82	206	108	105	105
query83	184	138	142	138
query84	245	58	53	53
query85	969	283	268	268
query86	448	368	385	368
query87	3538	3391	3383	3383
query88	3378	2286	2268	2268
query89	342	255	263	255
query90	1903	222	216	216
query91	122	92	95	92
query92	66	59	62	59
query93	2316	520	504	504
query94	854	197	190	190
query95	474	423	432	423
query96	628	321	320	320
query97	4247	4185	4218	4185
query98	216	202	200	200
query99	1121	797	848	797
Total cold run time: 293654 ms
Total hot run time: 179417 ms

doris-robot · 2023-12-30T12:52:54Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.07 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.6 seconds inserted 10000000 Rows, about 362K ops/s
storage size: 17183673893 Bytes

…) (apache#29300)

dataroaring approved these changes Dec 29, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 29, 2023

github-actionsBot added the reviewed label Dec 29, 2023

[Fix](schema change) disable convert light schema change (apache#28205)
a3a01a0

yiguolei force-pushed the luwei/disable-flush-col-id-master branch from 9cf4343 to a3a01a0Compare December 30, 2023 12:19

yiguolei approved these changes Dec 31, 2023
View reviewed changes

yiguolei merged commit 3c6c652 into apache:masterDec 31, 2023

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
[Fix](schema change) disable convert light schema change (apache#28205…
1a83e7a
…) (apache#29300)

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
[Fix](schema change) disable convert light schema change (apache#28205…
2722c45
…) (apache#29300)

Navigation Menu

Uh oh!

[Fix] (schema change) disable convert light schema change (#28205) - #29300

[Fix] (schema change) disable convert light schema change (#28205)#29300
yiguolei merged 1 commit into
apache:masterfrom
luwei16:luwei/disable-flush-col-id-master

luwei16 commented Dec 29, 2023

Uh oh!

luwei16 commented Dec 29, 2023

Uh oh!

dataroaring left a comment

Uh oh!

github-actionsBot commented Dec 29, 2023

Uh oh!

github-actionsBot commented Dec 29, 2023

Uh oh!

doris-robot commented Dec 29, 2023

Uh oh!

doris-robot commented Dec 29, 2023

Uh oh!

yiguolei commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

luwei16 commented Dec 29, 2023

Uh oh!

luwei16 commented Dec 29, 2023

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Dec 29, 2023

Uh oh!

github-actionsBot commented Dec 29, 2023

Uh oh!

doris-robot commented Dec 29, 2023

Uh oh!

doris-robot commented Dec 29, 2023

Uh oh!

yiguolei commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

doris-robot commented Dec 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants