[enhance](mtmv)mtmv not refresh cache of external table before run task by zddr · Pull Request #48172 · apache/doris

zddr · 2025-02-21T03:41:47Z

What problem does this PR solve?

before: after refresh MTMV, it will be sync with external

after: after refresh MTMV, it not necessarily fully synchronized with external， but it will be consistent with Doris' query of external

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
mtmv not refresh cache of external table before run task

Release note

mtmv not refresh cache of external table before run task

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

zddr · 2025-02-21T03:42:12Z

run buildall

hello-stephen · 2025-02-21T03:42:26Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

github-actions · 2025-02-21T03:46:49Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-21T03:46:51Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-21T04:17:39Z

TPC-H: Total hot run time: 31603 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 58ec4d59fd1ddb797237ce0005733fa8bb74155e, data reload: false
------ Round 1 ----------------------------------
q1	17586	5323	5146	5146
q2	2046	302	169	169
q3	10408	1221	741	741
q4	10204	1017	519	519
q5	7507	2448	2249	2249
q6	185	165	133	133
q7	924	790	606	606
q8	9314	1297	1130	1130
q9	4780	4662	4624	4624
q10	6801	2313	1876	1876
q11	479	278	254	254
q12	353	361	216	216
q13	17786	3699	3084	3084
q14	230	221	209	209
q15	526	478	461	461
q16	613	614	594	594
q17	580	862	331	331
q18	6915	6177	6235	6177
q19	1231	972	549	549
q20	315	328	189	189
q21	2880	2177	2039	2039
q22	375	353	307	307
Total cold run time: 102038 ms
Total hot run time: 31603 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5193	5187	5148	5148
q2	249	335	241	241
q3	2146	2698	2285	2285
q4	1430	1838	1382	1382
q5	4224	4090	4153	4090
q6	207	160	129	129
q7	1858	1850	1629	1629
q8	2591	2578	2555	2555
q9	7162	7230	7136	7136
q10	3016	3207	2775	2775
q11	571	515	488	488
q12	701	754	592	592
q13	3541	3868	3351	3351
q14	274	295	271	271
q15	505	475	478	475
q16	652	692	640	640
q17	1111	1595	1325	1325
q18	7512	7440	7292	7292
q19	785	830	864	830
q20	1979	2031	1919	1919
q21	5590	4996	4950	4950
q22	578	578	541	541
Total cold run time: 51875 ms
Total hot run time: 50044 ms

doris-robot · 2025-02-21T04:29:35Z

TPC-DS: Total hot run time: 190865 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 58ec4d59fd1ddb797237ce0005733fa8bb74155e, data reload: false
query1	1400	992	982	982
query2	6244	1893	1878	1878
query3	11114	4802	4544	4544
query4	54189	24719	23654	23654
query5	5158	544	483	483
query6	380	194	185	185
query7	5266	511	297	297
query8	321	243	231	231
query9	6577	2571	2585	2571
query10	400	308	267	267
query11	15150	15078	14873	14873
query12	155	110	107	107
query13	1182	509	395	395
query14	10267	7002	6703	6703
query15	203	190	185	185
query16	6990	671	450	450
query17	1062	711	565	565
query18	1518	413	321	321
query19	195	207	169	169
query20	125	121	120	120
query21	209	119	110	110
query22	4495	4647	4415	4415
query23	34050	33311	33394	33311
query24	5615	2446	2428	2428
query25	460	470	392	392
query26	648	271	155	155
query27	1739	507	330	330
query28	2923	2480	2429	2429
query29	581	560	425	425
query30	209	189	155	155
query31	877	893	841	841
query32	73	65	61	61
query33	435	363	289	289
query34	739	856	519	519
query35	818	836	744	744
query36	971	988	912	912
query37	119	97	71	71
query38	4397	4301	4254	4254
query39	1547	1445	1442	1442
query40	209	157	100	100
query41	55	50	50	50
query42	125	112	103	103
query43	531	511	489	489
query44	1277	809	802	802
query45	189	176	168	168
query46	883	1074	654	654
query47	1886	1897	1829	1829
query48	411	423	320	320
query49	719	493	424	424
query50	719	768	436	436
query51	4231	4280	4225	4225
query52	105	102	96	96
query53	238	263	186	186
query54	470	477	422	422
query55	80	87	87	87
query56	259	285	266	266
query57	1190	1231	1150	1150
query58	247	247	281	247
query59	3027	3071	2846	2846
query60	276	272	268	268
query61	126	116	132	116
query62	731	742	703	703
query63	229	196	197	196
query64	1418	1024	668	668
query65	3223	3103	3130	3103
query66	708	393	341	341
query67	15861	15486	15250	15250
query68	2700	790	529	529
query69	432	318	263	263
query70	1197	1087	1121	1087
query71	329	309	259	259
query72	6170	3423	3681	3423
query73	627	708	351	351
query74	9165	9175	8958	8958
query75	3143	3177	2671	2671
query76	1571	1151	751	751
query77	423	360	275	275
query78	10013	10265	9263	9263
query79	996	907	590	590
query80	613	543	482	482
query81	498	274	235	235
query82	652	128	101	101
query83	177	167	153	153
query84	290	94	79	79
query85	783	402	364	364
query86	323	311	303	303
query87	4446	4479	4461	4461
query88	2947	2216	2227	2216
query89	460	316	284	284
query90	1704	192	191	191
query91	130	134	110	110
query92	68	61	56	56
query93	1174	958	567	567
query94	492	410	296	296
query95	338	261	250	250
query96	496	527	264	264
query97	2767	2865	2779	2779
query98	217	207	262	207
query99	1349	1401	1239	1239
Total cold run time: 284914 ms
Total hot run time: 190865 ms

doris-robot · 2025-02-21T04:34:51Z

ClickBench: Total hot run time: 30.59 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 58ec4d59fd1ddb797237ce0005733fa8bb74155e, data reload: false
query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.42	0.41	0.41
query6	1.17	0.64	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.63	0.52	0.53
query10	0.58	0.58	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.63	0.60	0.60
query14	2.66	2.69	2.69
query15	0.94	0.85	0.85
query16	0.38	0.39	0.36
query17	1.02	1.01	1.06
query18	0.21	0.20	0.20
query19	1.87	1.86	1.98
query20	0.02	0.01	0.01
query21	15.36	0.91	0.60
query22	0.75	1.15	0.62
query23	15.01	1.39	0.63
query24	7.84	0.90	0.79
query25	0.51	0.20	0.07
query26	0.54	0.16	0.13
query27	0.05	0.05	0.05
query28	9.18	0.84	0.45
query29	12.58	4.00	3.36
query30	0.25	0.09	0.05
query31	2.84	0.57	0.39
query32	3.23	0.56	0.46
query33	2.99	3.00	3.01
query34	15.79	5.19	4.47
query35	4.63	4.55	4.53
query36	0.65	0.49	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.53 s
Total hot run time: 30.59 s

zddr · 2025-02-25T09:50:11Z

run buildall

doris-robot · 2025-02-25T10:10:58Z

TPC-H: Total hot run time: 31589 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d19dfc93fe93ea716864007a55d697b52d13aab, data reload: false
------ Round 1 ----------------------------------
q1	17618	5178	5091	5091
q2	2050	302	170	170
q3	10404	1280	737	737
q4	10217	1018	536	536
q5	7527	2340	2385	2340
q6	194	168	134	134
q7	905	760	616	616
q8	9327	1256	1127	1127
q9	5095	4828	4567	4567
q10	6825	2343	1874	1874
q11	476	288	261	261
q12	348	355	224	224
q13	17766	3682	3128	3128
q14	232	226	204	204
q15	518	486	466	466
q16	625	621	583	583
q17	555	838	352	352
q18	6766	6182	6170	6170
q19	1212	946	548	548
q20	335	322	200	200
q21	2772	2195	1965	1965
q22	367	342	296	296
Total cold run time: 102134 ms
Total hot run time: 31589 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5110	5098	5128	5098
q2	239	330	233	233
q3	2158	2685	2289	2289
q4	1427	1827	1353	1353
q5	4206	4151	4137	4137
q6	209	162	124	124
q7	1867	1842	1718	1718
q8	2614	2573	2517	2517
q9	7201	7165	6980	6980
q10	3013	3224	2757	2757
q11	588	524	510	510
q12	672	771	630	630
q13	3617	3972	3369	3369
q14	267	287	247	247
q15	498	451	446	446
q16	637	671	634	634
q17	1099	1534	1321	1321
q18	7168	7045	7046	7045
q19	788	796	911	796
q20	1883	1953	1787	1787
q21	5303	4816	4693	4693
q22	633	607	549	549
Total cold run time: 51197 ms
Total hot run time: 49233 ms

doris-robot · 2025-02-25T10:22:29Z

TPC-DS: Total hot run time: 184739 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d19dfc93fe93ea716864007a55d697b52d13aab, data reload: false
query1	989	380	383	380
query2	6570	1881	1884	1881
query3	6797	207	206	206
query4	26867	23389	23447	23389
query5	4461	665	514	514
query6	311	230	195	195
query7	4617	510	297	297
query8	295	256	234	234
query9	8620	2530	2516	2516
query10	491	323	267	267
query11	15586	15239	14929	14929
query12	165	114	113	113
query13	1690	526	384	384
query14	10793	6448	6747	6448
query15	223	191	182	182
query16	7653	657	474	474
query17	1601	753	568	568
query18	1990	423	320	320
query19	194	193	165	165
query20	128	122	115	115
query21	215	124	107	107
query22	4188	4300	4136	4136
query23	34034	33054	33022	33022
query24	7405	2377	2450	2377
query25	521	464	399	399
query26	1226	287	163	163
query27	2013	497	330	330
query28	3904	2409	2396	2396
query29	716	554	432	432
query30	239	187	159	159
query31	956	880	792	792
query32	79	68	64	64
query33	592	363	315	315
query34	793	846	518	518
query35	816	823	723	723
query36	966	1014	908	908
query37	116	103	72	72
query38	4065	4257	4169	4169
query39	1496	1382	1422	1382
query40	213	124	105	105
query41	55	56	53	53
query42	128	109	104	104
query43	487	507	496	496
query44	1281	796	799	796
query45	177	168	164	164
query46	875	1026	651	651
query47	1758	1776	1703	1703
query48	377	403	296	296
query49	802	489	441	441
query50	673	724	410	410
query51	4145	4244	4121	4121
query52	105	115	103	103
query53	239	268	190	190
query54	507	491	425	425
query55	91	82	80	80
query56	264	277	267	267
query57	1135	1160	1064	1064
query58	242	243	245	243
query59	2624	2723	2511	2511
query60	279	277	262	262
query61	125	118	123	118
query62	825	712	655	655
query63	240	199	190	190
query64	4285	1006	690	690
query65	3264	3162	3132	3132
query66	1054	420	368	368
query67	15554	15415	15237	15237
query68	8174	875	515	515
query69	475	309	264	264
query70	1225	1098	1084	1084
query71	467	298	273	273
query72	5519	3680	3721	3680
query73	798	734	352	352
query74	9133	8851	8905	8851
query75	3804	3165	2726	2726
query76	3692	1158	742	742
query77	785	387	289	289
query78	10014	10068	9346	9346
query79	2810	848	597	597
query80	607	523	514	514
query81	515	281	238	238
query82	708	127	96	96
query83	175	173	155	155
query84	247	94	77	77
query85	789	352	309	309
query86	398	309	304	304
query87	4422	4423	4378	4378
query88	3471	2206	2184	2184
query89	390	317	291	291
query90	1916	195	203	195
query91	139	142	113	113
query92	77	65	60	60
query93	1648	1070	584	584
query94	657	420	305	305
query95	355	271	258	258
query96	478	562	266	266
query97	3317	3388	3269	3269
query98	222	210	204	204
query99	1337	1416	1304	1304
Total cold run time: 274949 ms
Total hot run time: 184739 ms

doris-robot · 2025-02-25T10:27:50Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d19dfc93fe93ea716864007a55d697b52d13aab, data reload: false
query1	0.03	0.04	0.04
query2	0.06	0.03	0.03
query3	0.25	0.06	0.07
query4	1.61	0.10	0.10
query5	0.56	0.56	0.54
query6	1.22	0.72	0.72
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.52
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.82	2.69	2.83
query15	0.94	0.87	0.85
query16	0.37	0.38	0.38
query17	1.06	1.04	1.03
query18	0.22	0.20	0.21
query19	1.97	2.00	1.76
query20	0.02	0.01	0.02
query21	15.36	0.91	0.55
query22	0.75	1.07	0.63
query23	15.09	1.38	0.62
query24	6.91	1.51	0.69
query25	0.55	0.26	0.08
query26	0.63	0.17	0.13
query27	0.06	0.05	0.06
query28	9.87	0.80	0.44
query29	12.60	3.90	3.29
query30	0.25	0.09	0.07
query31	2.81	0.59	0.40
query32	3.25	0.56	0.48
query33	2.97	3.00	3.03
query34	15.79	5.13	4.49
query35	4.54	4.46	4.54
query36	0.67	0.49	0.49
query37	0.09	0.07	0.07
query38	0.05	0.03	0.04
query39	0.03	0.03	0.03
query40	0.18	0.13	0.12
query41	0.08	0.04	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.88 s
Total hot run time: 30.55 s

zddr · 2025-02-26T03:01:52Z

run buildall

doris-robot · 2025-02-26T03:37:30Z

TPC-H: Total hot run time: 32422 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20357c2ae307b9637b73ae5fe30ca2ba5b12e1bc, data reload: false
------ Round 1 ----------------------------------
q1	18104	5426	5137	5137
q2	2055	297	177	177
q3	12064	1262	772	772
q4	10291	1045	541	541
q5	8438	2446	2438	2438
q6	220	182	145	145
q7	977	795	634	634
q8	9808	1496	1247	1247
q9	5273	4880	5062	4880
q10	7932	2304	1920	1920
q11	500	287	266	266
q12	356	377	235	235
q13	18203	3809	3137	3137
q14	234	235	224	224
q15	517	485	454	454
q16	628	624	584	584
q17	601	904	371	371
q18	6634	6291	6091	6091
q19	2089	1043	593	593
q20	325	330	195	195
q21	2895	2268	2068	2068
q22	377	345	313	313
Total cold run time: 108521 ms
Total hot run time: 32422 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5509	5536	5363	5363
q2	243	336	230	230
q3	2164	2683	2335	2335
q4	1425	1847	1414	1414
q5	4250	4146	4340	4146
q6	236	185	133	133
q7	2007	1927	1719	1719
q8	2648	2602	2651	2602
q9	7120	7145	7300	7145
q10	3054	3232	2696	2696
q11	566	506	479	479
q12	695	776	634	634
q13	3557	3937	3271	3271
q14	281	307	272	272
q15	500	467	468	467
q16	669	675	644	644
q17	1196	1635	1341	1341
q18	7576	7424	7462	7424
q19	894	851	957	851
q20	2020	2035	1853	1853
q21	5544	4843	4789	4789
q22	686	630	544	544
Total cold run time: 52840 ms
Total hot run time: 50352 ms

doris-robot · 2025-02-26T03:49:38Z

TPC-DS: Total hot run time: 190650 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20357c2ae307b9637b73ae5fe30ca2ba5b12e1bc, data reload: false
query1	1340	937	941	937
query2	6157	1842	1862	1842
query3	11013	4691	4420	4420
query4	54094	24283	22926	22926
query5	5115	544	492	492
query6	351	195	188	188
query7	4961	506	304	304
query8	335	244	239	239
query9	6230	2614	2623	2614
query10	430	308	260	260
query11	15078	15001	14967	14967
query12	155	107	106	106
query13	1091	521	369	369
query14	10615	7139	6639	6639
query15	197	198	173	173
query16	7078	639	495	495
query17	1085	732	594	594
query18	1550	439	321	321
query19	202	203	167	167
query20	132	125	120	120
query21	213	128	107	107
query22	4534	4342	4332	4332
query23	34120	33449	33334	33334
query24	5934	2536	2500	2500
query25	491	504	423	423
query26	682	276	204	204
query27	1646	507	355	355
query28	2793	2502	2462	2462
query29	574	576	421	421
query30	221	205	157	157
query31	895	879	797	797
query32	76	62	63	62
query33	446	365	305	305
query34	764	869	506	506
query35	817	823	746	746
query36	952	1008	917	917
query37	119	103	75	75
query38	4161	4209	4179	4179
query39	1477	1454	1464	1454
query40	207	125	105	105
query41	51	50	49	49
query42	122	109	107	107
query43	517	540	487	487
query44	1353	834	820	820
query45	187	170	178	170
query46	896	1078	674	674
query47	1862	1832	1744	1744
query48	392	422	317	317
query49	709	530	429	429
query50	754	766	435	435
query51	4294	4288	4196	4196
query52	112	105	96	96
query53	233	255	192	192
query54	500	500	424	424
query55	92	86	85	85
query56	296	291	267	267
query57	1173	1184	1118	1118
query58	255	252	257	252
query59	2788	2718	2608	2608
query60	301	283	275	275
query61	121	123	116	116
query62	751	753	676	676
query63	240	192	192	192
query64	1493	1021	692	692
query65	3300	3245	3240	3240
query66	753	395	320	320
query67	15984	15721	15350	15350
query68	7255	883	518	518
query69	536	382	266	266
query70	1182	1109	1089	1089
query71	487	305	275	275
query72	5777	3674	3796	3674
query73	1407	735	374	374
query74	9009	8955	8998	8955
query75	3733	3169	2703	2703
query76	4241	1169	750	750
query77	611	380	278	278
query78	9996	10123	9283	9283
query79	2968	837	606	606
query80	729	558	457	457
query81	529	286	245	245
query82	705	127	98	98
query83	306	170	154	154
query84	292	90	74	74
query85	796	356	313	313
query86	399	295	275	275
query87	4451	4581	4244	4244
query88	3695	2238	2212	2212
query89	399	320	299	299
query90	1781	190	196	190
query91	140	145	111	111
query92	84	62	59	59
query93	2111	1077	578	578
query94	656	426	303	303
query95	350	279	260	260
query96	486	556	282	282
query97	3334	3372	3242	3242
query98	227	205	206	205
query99	1436	1415	1333	1333
Total cold run time: 297968 ms
Total hot run time: 190650 ms

doris-robot · 2025-02-26T03:54:56Z

ClickBench: Total hot run time: 30.93 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20357c2ae307b9637b73ae5fe30ca2ba5b12e1bc, data reload: false
query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.06	0.07
query4	1.62	0.10	0.10
query5	0.55	0.54	0.54
query6	1.21	0.72	0.72
query7	0.02	0.02	0.01
query8	0.03	0.04	0.04
query9	0.62	0.56	0.55
query10	0.59	0.62	0.59
query11	0.16	0.12	0.11
query12	0.16	0.13	0.11
query13	0.62	0.60	0.61
query14	2.71	2.78	2.72
query15	0.90	0.85	0.87
query16	0.36	0.38	0.37
query17	1.03	1.04	0.99
query18	0.21	0.20	0.19
query19	1.87	1.87	1.95
query20	0.01	0.01	0.01
query21	15.40	0.88	0.54
query22	0.74	1.22	0.74
query23	14.81	1.35	0.60
query24	7.35	1.55	0.99
query25	0.52	0.27	0.07
query26	0.59	0.17	0.14
query27	0.05	0.05	0.05
query28	9.79	0.90	0.42
query29	12.52	3.94	3.25
query30	0.25	0.09	0.07
query31	2.81	0.57	0.38
query32	3.23	0.54	0.46
query33	2.96	2.97	3.04
query34	15.65	5.12	4.49
query35	4.53	4.47	4.50
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.37 s
Total hot run time: 30.93 s

github-actions · 2025-02-27T04:01:54Z

PR approved by at least one committer and no changes requested.

…efore run async mv task (apache#48172) Problem Summary: before this PR, external catalog metadata will be sync when refresh async mv that based on external table. after this PR, remove sync metadata action, but the data in async mv still consistent with query in Doris on external table. metadata cache of external table no longer be refreshed before run async mv task

…refreshed b… (#48766) …efore run async mv task (#48172) pick: #48172

…refreshed b… (#48767) …efore run async mv task (#48172) pick: #48172

…50979) Problem Summary: related pr: #48172 This pr(#48172) had changed the code logical of method `beforeMTMVRefresh`, but this pr(#49956) added the code back. So we delete this code.

…efore run async mv task (apache#48172) ### What problem does this PR solve? Problem Summary: before this PR, external catalog metadata will be sync when refresh async mv that based on external table. after this PR, remove sync metadata action, but the data in async mv still consistent with query in Doris on external table. ### Release note metadata cache of external table no longer be refreshed before run async mv task

…pache#50979) Problem Summary: related pr: apache#48172 This pr(apache#48172) had changed the code logical of method `beforeMTMVRefresh`, but this pr(apache#49956) added the code back. So we delete this code.

…efore run async mv task (apache#48172) ### What problem does this PR solve? Problem Summary: before this PR, external catalog metadata will be sync when refresh async mv that based on external table. after this PR, remove sync metadata action, but the data in async mv still consistent with query in Doris on external table. ### Release note metadata cache of external table no longer be refreshed before run async mv task

…pache#50979) Problem Summary: related pr: apache#48172 This pr(apache#48172) had changed the code logical of method `beforeMTMVRefresh`, but this pr(apache#49956) added the code back. So we delete this code.

…efore run async mv task (apache#48172) Problem Summary: before this PR, external catalog metadata will be sync when refresh async mv that based on external table. after this PR, remove sync metadata action, but the data in async mv still consistent with query in Doris on external table. metadata cache of external table no longer be refreshed before run async mv task

…pache#50979) Problem Summary: related pr: apache#48172 This pr(apache#48172) had changed the code logical of method `beforeMTMVRefresh`, but this pr(apache#49956) added the code back. So we delete this code.

Cherry-pick from #43959 #44419 #44415 #44567 #44673 #44998 #45273 #44911 #44726 #45652 #45659 #46257 #46641 #47026 #47166 #48172 #49956 #50979 --------- Co-authored-by: James <lijibing@selectdb.com> Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>

1
58ec4d5

morrySnow previously approved these changes Feb 21, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 21, 2025

github-actionsBot added the reviewed label Feb 21, 2025

1
9d19dfc

zddr dismissed morrySnow’s stale review via 9d19dfcFebruary 25, 2025 09:49

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Feb 25, 2025

Merge branch 'master' into mv_not_refresh_cache
20357c2

morrySnow approved these changes Feb 27, 2025
View reviewed changes

morrySnow added p0_b dev/2.1.x dev/3.0.x labels Feb 27, 2025

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2025

seawinde approved these changes Mar 4, 2025
View reviewed changes

morrySnow merged commit 932b966 into apache:masterMar 5, 2025

github-actionsBot added dev/3.0.x-conflict dev/2.1.x-conflict labels Mar 5, 2025

zddr mentioned this pull request Mar 6, 2025
branch-2.1:[opt](mtmv) metadata cache of external table no longer be refreshed b… #48766
Merged

zddr mentioned this pull request Mar 6, 2025
branch-3.0:[opt](mtmv) metadata cache of external table no longer be refreshed b… #48767
Merged

yiguolei pushed a commit that referenced this pull request Mar 8, 2025
branch-2.1:[opt](mtmv) metadata cache of external table no longer be …
2b7e9c8
…refreshed b… (#48766) …efore run async mv task (#48172) pick: #48172

yiguolei added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Mar 8, 2025

dataroaring pushed a commit that referenced this pull request Mar 10, 2025
branch-3.0:[opt](mtmv) metadata cache of external table no longer be …
e71c5c0
…refreshed b… (#48767) …efore run async mv task (#48172) pick: #48172

dataroaring added dev/3.0.5-merged and removed dev/3.0.x dev/3.0.x-conflict labels Mar 10, 2025

BePPPower mentioned this pull request May 16, 2025
[Fix](MTMV) mtmv not refresh cache of external table before run task #50979
Merged
16 tasks

zddr mentioned this pull request Jun 20, 2025
branch-3.1:[enhance](mtmv)MTMV support iceberg/hudi/paimon #52040
Merged

Uh oh!

Conversation

zddr commented Feb 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

zddr commented Feb 21, 2025

Uh oh!

hello-stephen commented Feb 21, 2025

Uh oh!

github-actionsBot commented Feb 21, 2025

Uh oh!

github-actionsBot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

zddr commented Feb 25, 2025

Uh oh!

doris-robot commented Feb 25, 2025

Uh oh!

doris-robot commented Feb 25, 2025

Uh oh!

doris-robot commented Feb 25, 2025

Uh oh!

zddr commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

github-actionsBot commented Feb 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zddr commented Feb 21, 2025•
edited
Loading