[fix](Nereids) Use the schema saved during planning as the schema of the original target table by morrySnow · Pull Request #47337 · apache/doris

morrySnow · 2025-01-23T03:21:38Z

What problem does this PR solve?

Problem Summary:

because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-01-23T03:21:42Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morrySnow · 2025-01-23T03:22:03Z

run buildall

doris-robot · 2025-01-23T03:42:39Z

TPC-H: Total hot run time: 32366 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6d092cb485ac110442a087511ee898d5d2ca2dfc, data reload: false
------ Round 1 ----------------------------------
q1	17569	5594	5408	5408
q2	2043	307	179	179
q3	10511	1282	720	720
q4	10273	978	519	519
q5	8855	2438	2204	2204
q6	194	163	134	134
q7	910	769	622	622
q8	9251	1380	1203	1203
q9	5314	4895	4884	4884
q10	6849	2318	1882	1882
q11	476	274	264	264
q12	352	371	220	220
q13	17777	3775	3063	3063
q14	227	230	213	213
q15	503	483	456	456
q16	642	623	594	594
q17	584	871	335	335
q18	7178	6537	6433	6433
q19	2120	986	543	543
q20	312	331	187	187
q21	2867	2175	1982	1982
q22	365	333	321	321
Total cold run time: 105172 ms
Total hot run time: 32366 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5641	5499	5480	5480
q2	244	328	237	237
q3	2246	2655	2312	2312
q4	1391	1862	1377	1377
q5	4307	4734	4759	4734
q6	171	162	128	128
q7	2077	1995	1871	1871
q8	2654	2767	2658	2658
q9	7257	7262	7201	7201
q10	3023	3246	2829	2829
q11	590	526	497	497
q12	671	768	625	625
q13	3517	3887	3241	3241
q14	276	298	276	276
q15	503	477	458	458
q16	652	678	654	654
q17	1204	1735	1263	1263
q18	7644	7424	7228	7228
q19	815	913	1168	913
q20	1968	2022	1883	1883
q21	5781	5294	4962	4962
q22	611	585	559	559
Total cold run time: 53243 ms
Total hot run time: 51386 ms

doris-robot · 2025-01-23T03:54:27Z

TPC-DS: Total hot run time: 188036 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6d092cb485ac110442a087511ee898d5d2ca2dfc, data reload: false
query1	947	385	372	372
query2	6540	2048	1991	1991
query3	6786	218	214	214
query4	33110	23699	23581	23581
query5	4313	628	465	465
query6	296	198	195	195
query7	4618	506	315	315
query8	318	248	234	234
query9	9442	2699	2697	2697
query10	485	324	256	256
query11	17982	15171	15143	15143
query12	160	114	111	111
query13	1664	551	396	396
query14	9950	7162	6681	6681
query15	249	204	200	200
query16	8072	620	518	518
query17	1584	707	548	548
query18	2105	396	298	298
query19	219	197	181	181
query20	120	110	109	109
query21	216	126	100	100
query22	4384	4112	4234	4112
query23	34036	33231	32791	32791
query24	6469	2279	2251	2251
query25	490	447	391	391
query26	1224	269	154	154
query27	2027	466	335	335
query28	5253	2451	2420	2420
query29	738	570	416	416
query30	234	185	158	158
query31	959	885	800	800
query32	86	61	58	58
query33	509	374	317	317
query34	751	856	501	501
query35	806	830	757	757
query36	983	1066	941	941
query37	119	96	75	75
query38	4101	4075	4115	4075
query39	1464	1376	1407	1376
query40	202	116	101	101
query41	51	49	49	49
query42	137	112	108	108
query43	513	515	489	489
query44	1332	795	808	795
query45	179	172	166	166
query46	868	1041	640	640
query47	1787	1824	1752	1752
query48	374	407	313	313
query49	785	518	395	395
query50	631	657	410	410
query51	7032	6752	6893	6752
query52	104	103	92	92
query53	235	251	192	192
query54	485	482	419	419
query55	82	81	85	81
query56	272	289	250	250
query57	1171	1148	1090	1090
query58	251	238	267	238
query59	2922	3069	2877	2877
query60	284	271	271	271
query61	126	114	116	114
query62	792	697	642	642
query63	228	194	192	192
query64	4424	1064	646	646
query65	3267	3165	3196	3165
query66	1098	424	307	307
query67	15843	15677	15429	15429
query68	4630	815	529	529
query69	464	301	261	261
query70	1238	1164	1150	1150
query71	387	291	267	267
query72	6127	3841	3904	3841
query73	653	759	360	360
query74	10385	8983	9357	8983
query75	3187	3187	2660	2660
query76	3186	1190	777	777
query77	518	355	354	354
query78	10052	10071	9322	9322
query79	2378	836	624	624
query80	762	535	438	438
query81	512	278	247	247
query82	341	156	136	136
query83	179	174	151	151
query84	238	90	75	75
query85	761	365	297	297
query86	385	290	299	290
query87	4534	4459	4370	4370
query88	3801	2239	2196	2196
query89	404	335	301	301
query90	1701	193	193	193
query91	131	138	109	109
query92	62	56	54	54
query93	1195	876	531	531
query94	689	411	300	300
query95	332	263	255	255
query96	495	610	287	287
query97	2774	2886	2732	2732
query98	235	198	198	198
query99	1259	1352	1240	1240
Total cold run time: 283279 ms
Total hot run time: 188036 ms

doris-robot · 2025-01-23T03:59:47Z

ClickBench: Total hot run time: 31.04 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6d092cb485ac110442a087511ee898d5d2ca2dfc, data reload: false
query1	0.04	0.04	0.06
query2	0.08	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.42	0.44	0.41
query6	1.15	0.64	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.50	0.50
query10	0.56	0.57	0.55
query11	0.15	0.11	0.10
query12	0.14	0.10	0.11
query13	0.60	0.60	0.59
query14	2.80	2.76	2.84
query15	0.91	0.83	0.82
query16	0.40	0.39	0.37
query17	1.10	1.03	1.10
query18	0.22	0.20	0.20
query19	2.01	1.83	2.03
query20	0.01	0.02	0.01
query21	15.36	0.93	0.61
query22	0.75	0.88	0.69
query23	15.18	1.43	0.60
query24	2.58	1.20	1.93
query25	0.20	0.11	0.10
query26	0.32	0.15	0.13
query27	0.05	0.07	0.05
query28	14.39	1.01	0.44
query29	12.58	3.94	3.26
query30	0.25	0.08	0.06
query31	2.82	0.57	0.39
query32	3.22	0.54	0.46
query33	3.02	2.98	3.01
query34	16.84	5.24	4.58
query35	4.58	4.51	4.50
query36	0.65	0.51	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.39 s
Total hot run time: 31.04 s

…the original target table

morrySnow · 2025-01-23T14:07:10Z

run buildall

doris-robot · 2025-01-23T14:38:42Z

TPC-H: Total hot run time: 32136 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a01aeadd7974a606327f738edb76cdc2e91ace31, data reload: false
------ Round 1 ----------------------------------
q1	17566	5567	5373	5373
q2	2042	308	162	162
q3	10422	1224	746	746
q4	10211	948	515	515
q5	7508	2353	2118	2118
q6	192	178	137	137
q7	897	763	613	613
q8	9623	1380	1241	1241
q9	5314	4920	4994	4920
q10	6856	2319	1889	1889
q11	485	284	256	256
q12	339	349	220	220
q13	18049	3660	3067	3067
q14	228	226	201	201
q15	506	472	472	472
q16	618	607	598	598
q17	572	844	328	328
q18	7165	6339	6265	6265
q19	1515	940	537	537
q20	321	323	191	191
q21	2803	2220	1983	1983
q22	364	336	304	304
Total cold run time: 103596 ms
Total hot run time: 32136 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5497	5476	5435	5435
q2	237	325	235	235
q3	2225	2624	2294	2294
q4	1392	1818	1344	1344
q5	4314	4660	4626	4626
q6	167	156	126	126
q7	1996	1945	1862	1862
q8	2606	2823	2729	2729
q9	7358	7147	7247	7147
q10	3008	3267	2788	2788
q11	575	516	497	497
q12	661	714	584	584
q13	3544	4010	3315	3315
q14	278	302	283	283
q15	525	472	478	472
q16	632	678	651	651
q17	1230	1731	1262	1262
q18	7699	7564	7268	7268
q19	838	1060	1103	1060
q20	1978	2030	1898	1898
q21	5729	5205	4991	4991
q22	595	587	549	549
Total cold run time: 53084 ms
Total hot run time: 51416 ms

doris-robot · 2025-01-23T14:51:04Z

TPC-DS: Total hot run time: 192451 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a01aeadd7974a606327f738edb76cdc2e91ace31, data reload: false
query1	1286	943	918	918
query2	6283	2037	2037	2037
query3	11002	4402	4535	4402
query4	60409	30851	23762	23762
query5	5451	608	470	470
query6	416	190	180	180
query7	5494	505	314	314
query8	331	267	232	232
query9	8202	2584	2571	2571
query10	451	297	256	256
query11	17777	15170	15633	15170
query12	161	108	109	108
query13	1393	528	387	387
query14	11015	7149	7289	7149
query15	216	204	174	174
query16	7221	669	457	457
query17	1122	727	554	554
query18	1589	438	303	303
query19	199	184	159	159
query20	118	118	114	114
query21	209	123	105	105
query22	4400	4878	4437	4437
query23	33871	33249	33372	33249
query24	5531	2307	2386	2307
query25	468	457	406	406
query26	638	280	160	160
query27	1691	467	333	333
query28	3918	2486	2436	2436
query29	525	548	433	433
query30	213	181	159	159
query31	898	875	793	793
query32	69	58	57	57
query33	431	358	293	293
query34	717	884	514	514
query35	811	838	761	761
query36	1001	1065	959	959
query37	122	103	76	76
query38	4300	4376	4381	4376
query39	1497	1440	1439	1439
query40	222	118	108	108
query41	55	56	54	54
query42	128	107	109	107
query43	520	537	503	503
query44	1336	841	845	841
query45	220	168	168	168
query46	869	1047	666	666
query47	1932	1942	1873	1873
query48	389	411	311	311
query49	697	487	423	423
query50	655	665	394	394
query51	4299	4354	4311	4311
query52	103	102	94	94
query53	245	262	187	187
query54	481	513	431	431
query55	80	85	77	77
query56	266	257	247	247
query57	1216	1231	1138	1138
query58	234	227	231	227
query59	3104	3212	3011	3011
query60	280	289	268	268
query61	117	115	114	114
query62	741	722	629	629
query63	217	188	188	188
query64	1161	1055	656	656
query65	3234	3167	3161	3161
query66	707	389	309	309
query67	15677	15573	15561	15561
query68	2204	842	549	549
query69	410	307	269	269
query70	1201	1129	1156	1129
query71	345	291	261	261
query72	5121	3927	4042	3927
query73	646	758	365	365
query74	9634	9131	8766	8766
query75	3161	3195	2640	2640
query76	1747	1182	795	795
query77	362	373	284	284
query78	10175	10081	9347	9347
query79	2743	809	607	607
query80	1909	572	445	445
query81	565	274	232	232
query82	842	146	123	123
query83	231	166	153	153
query84	277	89	76	76
query85	788	348	305	305
query86	413	323	300	300
query87	4471	4596	4430	4430
query88	3783	2182	2149	2149
query89	395	329	293	293
query90	1784	287	184	184
query91	137	136	106	106
query92	57	58	53	53
query93	1841	910	532	532
query94	689	424	303	303
query95	325	267	254	254
query96	487	600	290	290
query97	2802	2893	2760	2760
query98	223	205	189	189
query99	1279	1381	1268	1268
Total cold run time: 301678 ms
Total hot run time: 192451 ms

doris-robot · 2025-01-23T14:56:22Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a01aeadd7974a606327f738edb76cdc2e91ace31, data reload: false
query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.11
query5	0.42	0.42	0.40
query6	1.17	0.66	0.65
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.50
query10	0.56	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.12	0.11
query13	0.60	0.60	0.60
query14	2.84	2.80	2.84
query15	0.90	0.83	0.84
query16	0.37	0.39	0.37
query17	1.03	1.00	1.06
query18	0.23	0.21	0.21
query19	1.96	1.80	1.96
query20	0.01	0.01	0.02
query21	15.37	0.95	0.58
query22	0.76	0.83	0.82
query23	15.12	1.41	0.57
query24	3.38	0.73	1.41
query25	0.12	0.11	0.21
query26	0.31	0.14	0.14
query27	0.08	0.06	0.04
query28	13.79	1.03	0.45
query29	12.56	3.94	3.26
query30	0.25	0.09	0.06
query31	2.82	0.61	0.38
query32	3.23	0.56	0.46
query33	2.99	3.03	3.06
query34	16.57	5.11	4.44
query35	4.54	4.48	4.54
query36	0.64	0.50	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 106.01 s
Total hot run time: 30.55 s

github-actions · 2025-01-24T04:13:38Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-01-24T04:13:40Z

PR approved by anyone and no changes requested.

…the original target table (#47337) ### What problem does this PR solve? Related PR: #47033 #45045 Problem Summary: because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

…the original target table (#47337) Related PR: #47033 #45045 Problem Summary: because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

…e schema of the original target table #47337 (#47402) Cherry-picked from #47337 Co-authored-by: morrySnow <zhangwenxin@selectdb.com>

…e schema of the original target table #47337 pick from master #47337 Related PR: #47033 #45045 Problem Summary: because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

…e schema of the original target table #47337 (#47773) pick from master #47337

…e schema of the original target table apache#47337 (apache#47773) pick from master apache#47337

…the original target table (apache#47337) ### What problem does this PR solve? Related PR: apache#47033 apache#45045 Problem Summary: because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

…e schema of the original target table pick from master apache#47337

…the original target table (apache#47337) ### What problem does this PR solve? Related PR: apache#47033 apache#45045 Problem Summary: because schema change does not involve recreating the table object, but rather rebuilding the full schema. So, we should use the schema saved during planning as the schema of the original target table.

morrySnow added dev/2.1.x dev/3.0.x p0_b labels Jan 23, 2025

[fix](Nereids) Use the schema saved during planning as the schema of …
a01aead
…the original target table

morrySnow force-pushed the fix_lock branch from 6d092cb to a01aeadCompare January 23, 2025 14:06

morrySnow requested a review from 924060929 January 24, 2025 03:46

924060929 approved these changes Jan 24, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jan 24, 2025

github-actionsBot added the reviewed label Jan 24, 2025

feiniaofeiafei approved these changes Jan 24, 2025
View reviewed changes

morrySnow merged commit 4aaa657 into apache:masterJan 24, 2025

morrySnow deleted the fix_lock branch January 24, 2025 06:57

github-actionsBot mentioned this pull request Jan 24, 2025
branch-3.0: [fix](Nereids) Use the schema saved during planning as the schema of the original target table #47337 #47402
Merged

github-actionsBot added the dev/2.1.x-conflict label Jan 24, 2025

dataroaring pushed a commit that referenced this pull request Jan 26, 2025
branch-3.0: [fix](Nereids) Use the schema saved during planning as th…
0f897ed
…e schema of the original target table #47337 (#47402) Cherry-picked from #47337 Co-authored-by: morrySnow <zhangwenxin@selectdb.com>

dataroaring added the dev/3.0.4-merged label Jan 26, 2025

dataroaring removed the dev/3.0.x label Jan 26, 2025

morrySnow mentioned this pull request Feb 11, 2025
branch-2.1: [fix](Nereids) Use the schema saved during planning as the schema of the original target table #47337 #47773
Merged

yiguolei pushed a commit that referenced this pull request Feb 12, 2025
branch-2.1: [fix](Nereids) Use the schema saved during planning as th…
3f1a9db
…e schema of the original target table #47337 (#47773) pick from master #47337

yiguolei added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Feb 12, 2025

morningman pushed a commit to morningman/doris that referenced this pull request Feb 20, 2025
branch-2.1: [fix](Nereids) Use the schema saved during planning as th…
6c81bb9
…e schema of the original target table apache#47337 (apache#47773) pick from master apache#47337

deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Apr 30, 2025
branch-2.1: [fix](Nereids) Use the schema saved during planning as th…
c968c81
…e schema of the original target table pick from master apache#47337

Navigation Menu

Uh oh!

[fix](Nereids) Use the schema saved during planning as the schema of the original target table - #47337

[fix](Nereids) Use the schema saved during planning as the schema of the original target table#47337
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix_lock

morrySnow commented Jan 23, 2025

Uh oh!

Thearas commented Jan 23, 2025

Uh oh!

morrySnow commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

morrySnow commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

github-actionsBot commented Jan 24, 2025

Uh oh!

github-actionsBot commented Jan 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

morrySnow commented Jan 23, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jan 23, 2025

Uh oh!

morrySnow commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

morrySnow commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

doris-robot commented Jan 23, 2025

Uh oh!

github-actionsBot commented Jan 24, 2025

Uh oh!

github-actionsBot commented Jan 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants