[fix](hudi)Add hudi catalog read partition table partition prune by hubgeter · Pull Request #44669 · apache/doris

hubgeter · 2024-11-27T04:47:04Z

What problem does this PR solve?

Problem Summary:
In the previous PR #41218, some partition pruning logic was changed, which caused the hudi partition pruning to fail. This PR is to fix this problem.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-27T04:47:09Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hubgeter · 2024-12-01T14:54:09Z

run buildall

doris-robot · 2024-12-01T15:13:12Z

TPC-H: Total hot run time: 40197 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0dd819c30c083e7b52e4d5d7f4ff19139443d0f0, data reload: false
------ Round 1 ----------------------------------
q1	17602	7992	7286	7286
q2	2038	183	177	177
q3	10541	1168	1171	1168
q4	10226	767	679	679
q5	7601	2778	2715	2715
q6	247	147	149	147
q7	1008	646	609	609
q8	9338	1920	1951	1920
q9	7403	6557	6492	6492
q10	7094	2362	2305	2305
q11	475	262	249	249
q12	524	221	235	221
q13	17793	3048	3032	3032
q14	245	217	211	211
q15	571	511	514	511
q16	664	565	568	565
q17	1001	653	560	560
q18	7360	6685	6643	6643
q19	1327	1028	1008	1008
q20	495	173	184	173
q21	4086	3305	3217	3217
q22	383	321	309	309
Total cold run time: 108022 ms
Total hot run time: 40197 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7230	7220	7219	7219
q2	322	234	230	230
q3	2884	2823	2799	2799
q4	1970	1754	1700	1700
q5	5418	5398	5468	5398
q6	218	136	136	136
q7	2126	1676	1686	1676
q8	3234	3390	3457	3390
q9	8656	8724	8637	8637
q10	3498	3500	3448	3448
q11	601	483	500	483
q12	804	584	579	579
q13	13860	3009	3031	3009
q14	292	255	270	255
q15	557	517	516	516
q16	688	651	637	637
q17	1811	1594	1586	1586
q18	7869	7293	7383	7293
q19	1665	1643	1444	1444
q20	2081	1834	1817	1817
q21	5477	5220	5262	5220
q22	612	569	572	569
Total cold run time: 71873 ms
Total hot run time: 58041 ms

doris-robot · 2024-12-01T15:24:54Z

TPC-DS: Total hot run time: 191298 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0dd819c30c083e7b52e4d5d7f4ff19139443d0f0, data reload: false
query1	863	405	372	372
query2	5197	2112	2090	2090
query3	3866	206	214	206
query4	33495	23597	23599	23597
query5	2804	462	448	448
query6	229	180	185	180
query7	3242	327	316	316
query8	277	231	239	231
query9	8068	2765	2760	2760
query10	427	264	259	259
query11	17815	15243	15278	15243
query12	149	103	109	103
query13	1247	418	413	413
query14	8715	7631	7087	7087
query15	313	180	192	180
query16	7017	478	457	457
query17	1024	577	585	577
query18	2005	283	286	283
query19	355	153	148	148
query20	123	108	110	108
query21	209	100	109	100
query22	4542	4264	4332	4264
query23	35020	34171	34177	34171
query24	10194	2522	2459	2459
query25	644	379	379	379
query26	1662	149	153	149
query27	2728	291	288	288
query28	7681	2453	2455	2453
query29	755	413	403	403
query30	257	148	145	145
query31	1003	818	830	818
query32	68	55	86	55
query33	600	315	294	294
query34	898	514	538	514
query35	913	739	723	723
query36	1101	945	971	945
query37	116	72	76	72
query38	4502	4264	4413	4264
query39	1448	1431	1411	1411
query40	281	99	99	99
query41	46	41	42	41
query42	110	98	99	98
query43	553	491	500	491
query44	1219	828	819	819
query45	189	169	172	169
query46	1158	715	704	704
query47	1945	1840	1830	1830
query48	398	320	311	311
query49	846	388	384	384
query50	819	386	400	386
query51	7173	7147	7115	7115
query52	98	89	87	87
query53	251	179	183	179
query54	725	401	397	397
query55	80	92	72	72
query56	256	223	221	221
query57	1247	1117	1119	1117
query58	219	206	209	206
query59	3208	2935	2855	2855
query60	263	258	249	249
query61	108	108	105	105
query62	869	673	700	673
query63	216	190	184	184
query64	4210	663	645	645
query65	3293	3202	3242	3202
query66	934	310	311	310
query67	15914	15478	15585	15478
query68	4922	566	548	548
query69	416	257	251	251
query70	1222	1147	1154	1147
query71	310	260	241	241
query72	5927	4137	4097	4097
query73	756	359	371	359
query74	10264	8968	9004	8968
query75	3411	2682	2663	2663
query76	1724	1082	968	968
query77	437	273	267	267
query78	10255	9350	9325	9325
query79	1116	601	603	601
query80	1027	476	432	432
query81	531	237	232	232
query82	202	119	117	117
query83	169	148	152	148
query84	261	68	71	68
query85	1130	307	313	307
query86	362	266	314	266
query87	4760	4511	4595	4511
query88	3568	2255	2212	2212
query89	399	304	298	298
query90	2165	195	190	190
query91	136	103	103	103
query92	64	51	51	51
query93	1113	548	551	548
query94	1229	287	297	287
query95	353	267	247	247
query96	613	285	277	277
query97	2845	2682	2664	2664
query98	221	199	190	190
query99	1530	1336	1323	1323
Total cold run time: 281638 ms
Total hot run time: 191298 ms

doris-robot · 2024-12-01T15:30:24Z

ClickBench: Total hot run time: 32.56 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0dd819c30c083e7b52e4d5d7f4ff19139443d0f0, data reload: false
query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.60	0.10	0.11
query5	0.41	0.40	0.44
query6	1.15	0.68	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.50	0.50
query10	0.55	0.56	0.56
query11	0.15	0.11	0.11
query12	0.13	0.11	0.11
query13	0.62	0.60	0.60
query14	2.70	2.84	2.76
query15	0.89	0.83	0.82
query16	0.40	0.39	0.40
query17	1.07	1.06	1.00
query18	0.21	0.21	0.21
query19	1.96	1.87	2.06
query20	0.01	0.01	0.01
query21	15.36	0.59	0.62
query22	2.78	3.08	1.45
query23	17.05	0.88	0.92
query24	2.92	1.14	1.75
query25	0.31	0.12	0.09
query26	0.53	0.13	0.14
query27	0.04	0.06	0.04
query28	10.16	1.10	1.07
query29	12.56	3.22	3.22
query30	0.25	0.07	0.07
query31	2.86	0.39	0.38
query32	3.26	0.47	0.47
query33	2.94	3.03	3.01
query34	16.90	4.46	4.46
query35	4.52	4.51	4.50
query36	0.66	0.48	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.18	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.45 s
Total hot run time: 32.56 s

Co-authored-by: Socrates <suyiteng@selectdb.com>

hubgeter · 2024-12-03T09:36:18Z

run buildall

doris-robot · 2024-12-03T09:55:24Z

TPC-H: Total hot run time: 40035 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03d37805ed2903317b74111dc11bc27271f95162, data reload: false
------ Round 1 ----------------------------------
q1	17650	7446	7290	7290
q2	2044	186	187	186
q3	10524	1119	1169	1119
q4	10215	753	787	753
q5	7592	2686	2641	2641
q6	233	146	151	146
q7	996	624	608	608
q8	9249	1848	1881	1848
q9	6671	6463	6447	6447
q10	6955	2259	2308	2259
q11	465	254	258	254
q12	423	224	225	224
q13	17775	3033	3112	3033
q14	239	216	211	211
q15	579	546	513	513
q16	649	576	583	576
q17	974	531	533	531
q18	7294	6637	6673	6637
q19	1334	1020	1017	1017
q20	474	182	177	177
q21	4008	3273	3254	3254
q22	375	311	312	311
Total cold run time: 106718 ms
Total hot run time: 40035 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7230	7265	7281	7265
q2	331	227	234	227
q3	2896	2781	3017	2781
q4	2142	1817	1806	1806
q5	5877	5707	5613	5613
q6	221	139	137	137
q7	2199	1750	1750	1750
q8	3342	3535	3485	3485
q9	8942	9001	8962	8962
q10	3591	3560	3514	3514
q11	594	510	498	498
q12	829	629	629	629
q13	13203	3231	3186	3186
q14	301	283	282	282
q15	561	528	525	525
q16	679	642	636	636
q17	1806	1587	1580	1580
q18	7720	7538	7393	7393
q19	1645	1537	1535	1535
q20	2068	1824	1811	1811
q21	5316	5236	5231	5231
q22	646	574	556	556
Total cold run time: 72139 ms
Total hot run time: 59402 ms

doris-robot · 2024-12-03T10:07:25Z

TPC-DS: Total hot run time: 190053 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03d37805ed2903317b74111dc11bc27271f95162, data reload: false
query1	958	378	372	372
query2	6514	2025	1944	1944
query3	6704	207	209	207
query4	34173	23517	23493	23493
query5	4354	458	429	429
query6	253	176	162	162
query7	4621	308	300	300
query8	288	237	226	226
query9	9612	2673	2671	2671
query10	478	260	252	252
query11	18171	15251	15223	15223
query12	165	108	108	108
query13	1684	431	388	388
query14	9913	7311	6806	6806
query15	264	179	175	175
query16	8120	473	450	450
query17	1616	570	537	537
query18	2146	283	287	283
query19	343	146	144	144
query20	128	111	115	111
query21	206	100	100	100
query22	4572	4259	4323	4259
query23	34819	34032	34150	34032
query24	10425	2475	2451	2451
query25	648	368	363	363
query26	1395	143	151	143
query27	2870	274	274	274
query28	8292	2416	2398	2398
query29	875	395	414	395
query30	287	148	146	146
query31	1049	808	829	808
query32	98	55	56	55
query33	783	298	299	298
query34	960	512	564	512
query35	911	762	769	762
query36	1090	967	939	939
query37	135	71	72	71
query38	4301	4234	4254	4234
query39	1465	1426	1437	1426
query40	276	98	98	98
query41	45	44	43	43
query42	110	96	96	96
query43	535	486	465	465
query44	1264	795	785	785
query45	186	161	169	161
query46	1162	719	695	695
query47	1977	1871	1843	1843
query48	399	301	314	301
query49	1170	377	382	377
query50	820	382	387	382
query51	7314	7107	6990	6990
query52	103	93	89	89
query53	255	179	187	179
query54	1272	404	404	404
query55	83	74	80	74
query56	266	249	249	249
query57	1253	1101	1092	1092
query58	231	225	222	222
query59	3176	2997	2881	2881
query60	288	261	272	261
query61	135	130	132	130
query62	913	677	713	677
query63	216	184	183	183
query64	5491	658	624	624
query65	3319	3229	3237	3229
query66	1449	307	319	307
query67	15998	15803	15377	15377
query68	4489	565	540	540
query69	433	251	249	249
query70	1201	1024	1142	1024
query71	342	243	242	242
query72	6479	4058	4057	4057
query73	767	360	360	360
query74	10142	8999	9043	8999
query75	3490	2642	2645	2642
query76	2915	1057	1109	1057
query77	409	272	287	272
query78	10320	9478	9838	9478
query79	1569	584	584	584
query80	1123	431	450	431
query81	521	291	228	228
query82	955	119	117	117
query83	237	149	139	139
query84	235	66	70	66
query85	1274	296	295	295
query86	375	304	265	265
query87	4737	4611	4506	4506
query88	3170	2203	2163	2163
query89	393	294	293	293
query90	1941	184	189	184
query91	138	104	100	100
query92	59	49	49	49
query93	1105	526	533	526
query94	925	287	296	287
query95	346	253	245	245
query96	619	276	275	275
query97	2840	2705	2701	2701
query98	225	196	197	196
query99	1576	1359	1322	1322
Total cold run time: 300742 ms
Total hot run time: 190053 ms

doris-robot · 2024-12-03T10:12:56Z

ClickBench: Total hot run time: 33.25 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03d37805ed2903317b74111dc11bc27271f95162, data reload: false
query1	0.03	0.04	0.03
query2	0.08	0.02	0.04
query3	0.23	0.07	0.07
query4	1.63	0.10	0.10
query5	0.43	0.40	0.40
query6	1.18	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.57	0.51	0.51
query10	0.55	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.82	2.71	2.83
query15	0.92	0.83	0.83
query16	0.40	0.38	0.38
query17	1.05	1.02	1.04
query18	0.21	0.20	0.21
query19	1.85	1.88	2.00
query20	0.01	0.01	0.01
query21	15.36	0.60	0.58
query22	2.89	1.67	2.20
query23	17.12	0.90	0.96
query24	2.79	1.59	2.31
query25	0.36	0.09	0.10
query26	0.49	0.14	0.14
query27	0.04	0.04	0.04
query28	9.26	1.10	1.07
query29	12.59	3.25	3.23
query30	0.26	0.06	0.06
query31	2.88	0.38	0.38
query32	3.26	0.46	0.46
query33	3.04	3.11	3.08
query34	17.19	4.46	4.46
query35	4.57	4.54	4.51
query36	0.66	0.48	0.47
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.16 s
Total hot run time: 33.25 s

github-actions · 2024-12-04T16:24:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-12-04T16:24:28Z

PR approved by anyone and no changes requested.

hubgeter · 2024-12-05T03:21:26Z

wait me test use_hive_sync_partition = false in hudi 0.15

hubgeter · 2024-12-05T12:13:09Z

run buildall

hubgeter · 2024-12-05T12:15:25Z

wait me test use_hive_sync_partition = false in hudi 0.15

done. append test : use_hive_sync_partition = false/true

doris-robot · 2024-12-05T12:52:15Z

TPC-H: Total hot run time: 40351 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f213fcf28823bfaf35d6231e064595c1e36147b7, data reload: false
------ Round 1 ----------------------------------
q1	17586	7547	7348	7348
q2	2047	184	172	172
q3	10585	1087	1158	1087
q4	10575	754	751	751
q5	7599	2744	2744	2744
q6	234	150	151	150
q7	986	640	614	614
q8	9251	1874	1943	1874
q9	6638	6541	6434	6434
q10	6938	2283	2342	2283
q11	451	267	266	266
q12	450	230	220	220
q13	17774	3013	3058	3013
q14	250	226	221	221
q15	590	537	526	526
q16	660	605	612	605
q17	1001	522	561	522
q18	7193	6790	6841	6790
q19	1340	1012	979	979
q20	478	185	180	180
q21	4102	3824	3258	3258
q22	419	318	314	314
Total cold run time: 107147 ms
Total hot run time: 40351 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7423	7343	7299	7299
q2	325	231	227	227
q3	2892	2931	2926	2926
q4	2168	1900	1898	1898
q5	5624	5648	5658	5648
q6	225	148	143	143
q7	2208	1839	1812	1812
q8	3374	3551	3516	3516
q9	8915	8985	9051	8985
q10	3619	3585	3553	3553
q11	607	519	499	499
q12	802	652	645	645
q13	11755	3266	3234	3234
q14	320	272	289	272
q15	574	536	505	505
q16	683	633	644	633
q17	1866	1661	1612	1612
q18	8333	7770	7599	7599
q19	1686	1573	1596	1573
q20	2185	1887	1869	1869
q21	5725	5506	5624	5506
q22	666	551	576	551
Total cold run time: 71975 ms
Total hot run time: 60505 ms

doris-robot · 2024-12-05T13:04:56Z

TPC-DS: Total hot run time: 197168 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f213fcf28823bfaf35d6231e064595c1e36147b7, data reload: false
query1	1438	972	972	972
query2	6261	2058	2024	2024
query3	10922	4389	4178	4178
query4	67034	28325	23591	23591
query5	4942	470	434	434
query6	436	187	180	180
query7	5492	303	299	299
query8	333	243	240	240
query9	8575	2728	2714	2714
query10	414	255	234	234
query11	17166	15403	15952	15403
query12	179	105	111	105
query13	1437	435	397	397
query14	9799	6686	6884	6686
query15	222	186	184	184
query16	7333	478	498	478
query17	1073	583	595	583
query18	1813	328	317	317
query19	215	169	157	157
query20	121	118	119	118
query21	218	110	105	105
query22	4765	4576	4442	4442
query23	35660	34672	35270	34672
query24	5533	2488	2489	2488
query25	500	369	396	369
query26	697	152	163	152
query27	2543	286	287	286
query28	4673	2482	2463	2463
query29	683	437	422	422
query30	256	148	150	148
query31	1017	838	833	833
query32	65	55	57	55
query33	424	288	283	283
query34	914	533	526	526
query35	927	775	738	738
query36	1095	968	999	968
query37	131	73	71	71
query38	4537	4350	4444	4350
query39	1504	1489	1471	1471
query40	212	96	98	96
query41	43	40	47	40
query42	117	102	100	100
query43	559	500	510	500
query44	1152	811	812	811
query45	191	184	170	170
query46	1165	708	746	708
query47	2076	1929	1944	1929
query48	427	313	348	313
query49	733	390	374	374
query50	857	396	395	395
query51	7432	7121	7000	7000
query52	96	85	89	85
query53	261	174	182	174
query54	509	376	395	376
query55	80	74	74	74
query56	246	242	257	242
query57	1223	1101	1133	1101
query58	215	213	209	209
query59	3344	3367	3311	3311
query60	267	249	243	243
query61	107	103	106	103
query62	825	668	689	668
query63	212	186	184	184
query64	1923	658	630	630
query65	3267	3203	3210	3203
query66	750	296	302	296
query67	16077	15581	15546	15546
query68	3714	580	562	562
query69	435	256	248	248
query70	1217	1135	1156	1135
query71	398	244	246	244
query72	6140	4057	3986	3986
query73	781	356	355	355
query74	10126	9162	9106	9106
query75	3414	2640	2672	2640
query76	1962	1065	1167	1065
query77	468	324	283	283
query78	10430	9504	9494	9494
query79	1793	585	597	585
query80	1370	414	450	414
query81	512	226	244	226
query82	1314	125	113	113
query83	257	148	152	148
query84	280	70	69	69
query85	1036	301	300	300
query86	418	300	278	278
query87	4694	4699	4601	4601
query88	3797	2233	2177	2177
query89	412	308	288	288
query90	1993	186	180	180
query91	137	105	105	105
query92	62	48	50	48
query93	2379	545	541	541
query94	831	272	269	269
query95	354	244	246	244
query96	615	280	271	271
query97	2882	2707	2696	2696
query98	217	190	187	187
query99	1777	1332	1293	1293
Total cold run time: 321928 ms
Total hot run time: 197168 ms

doris-robot · 2024-12-05T13:10:26Z

ClickBench: Total hot run time: 33.15 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f213fcf28823bfaf35d6231e064595c1e36147b7, data reload: false
query1	0.04	0.03	0.02
query2	0.07	0.04	0.03
query3	0.24	0.07	0.06
query4	1.63	0.10	0.10
query5	0.42	0.39	0.41
query6	1.15	0.67	0.66
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.52	0.51
query10	0.56	0.56	0.58
query11	0.15	0.11	0.10
query12	0.14	0.11	0.12
query13	0.61	0.61	0.59
query14	2.74	2.72	2.86
query15	0.90	0.83	0.83
query16	0.39	0.38	0.39
query17	1.06	1.04	0.98
query18	0.23	0.21	0.20
query19	1.88	1.83	1.98
query20	0.01	0.01	0.02
query21	15.36	0.58	0.57
query22	2.62	1.90	2.22
query23	17.11	0.90	0.81
query24	3.31	1.55	2.00
query25	0.21	0.29	0.10
query26	0.48	0.14	0.14
query27	0.05	0.05	0.04
query28	9.58	1.12	1.09
query29	12.55	3.30	3.27
query30	0.25	0.06	0.06
query31	2.90	0.38	0.40
query32	3.24	0.47	0.45
query33	3.09	3.00	3.01
query34	17.21	4.43	4.49
query35	4.56	4.49	4.46
query36	0.65	0.47	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.5 s
Total hot run time: 33.15 s

github-actions · 2024-12-05T18:00:06Z

PR approved by at least one committer and no changes requested.

suxiaogang223

+1

) ### What problem does this PR solve? Problem Summary: In the previous PR #41218, some partition pruning logic was changed, which caused the hudi partition pruning to fail. This PR is to fix this problem. ### Release note [fix](hudi) fix hudi partition prune issue

…n prune #44669 (#45111) Cherry-picked from #44669 Co-authored-by: daidai <changyuwei@selectdb.com>

…n prune #44669 (#45113) Cherry-picked from #44669 Co-authored-by: daidai <changyuwei@selectdb.com>

…che#44669) ### What problem does this PR solve? Problem Summary: In the previous PR apache#41218, some partition pruning logic was changed, which caused the hudi partition pruning to fail. This PR is to fix this problem. ### Release note [fix](hudi) fix hudi partition prune issue

hubgeter changed the title ~~[enchement](hudi)Add hudi catalog read partition table partition prune~~[fix](hudi)Add hudi catalog read partition table partition pruneNov 27, 2024

morningman added usercase Important user case type label dev/2.1.x dev/3.0.x labels Nov 27, 2024

[enchement](hudi)Add hudi catalog read partition table partition prune
0dd819c

hubgeter force-pushed the hudi_prune_partition branch from d324c82 to 0dd819cCompare December 1, 2024 14:53

hubgeter added 2 commits December 3, 2024 17:31

append some test
816c70f
Co-authored-by: Socrates <suyiteng@selectdb.com>

fix dmp
03d3780

Merge branch 'master' into hudi_prune_partition
51d5fba

morningman previously approved these changes Dec 4, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 4, 2024

hubgeter added 2 commits December 5, 2024 15:42

fix compile
080f3e5

fix case.
f213fcf

hubgeter dismissed morningman’s stale review via f213fcfDecember 5, 2024 12:12

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Dec 5, 2024

morningman approved these changes Dec 5, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 5, 2024

suxiaogang223 approved these changes Dec 6, 2024
View reviewed changes

morningman merged commit 689bd37 into apache:masterDec 6, 2024

github-actionsBot mentioned this pull request Dec 6, 2024
branch-3.0: [fix](hudi)Add hudi catalog read partition table partition prune #44669 #45111
Merged

github-actionsBot mentioned this pull request Dec 6, 2024
branch-2.1: [fix](hudi)Add hudi catalog read partition table partition prune #44669 #45113
Merged

morningman pushed a commit that referenced this pull request Dec 6, 2024
branch-3.0: [fix](hudi)Add hudi catalog read partition table partitio…
7df5c10
…n prune #44669 (#45111) Cherry-picked from #44669 Co-authored-by: daidai <changyuwei@selectdb.com>

morningman added dev/3.0.4-merged and removed dev/3.0.x labels Dec 6, 2024

morningman pushed a commit that referenced this pull request Dec 6, 2024
branch-2.1: [fix](hudi)Add hudi catalog read partition table partitio…
b4e2e2c
…n prune #44669 (#45113) Cherry-picked from #44669 Co-authored-by: daidai <changyuwei@selectdb.com>

morningman added dev/2.1.8-merged and removed dev/2.1.x labels Dec 6, 2024

yiguolei mentioned this pull request Jan 19, 2025
Release Note 2.1.8 #47198
Closed

gavinchou mentioned this pull request Feb 18, 2025
Release Notes 3.0.4 #48013
Closed

Uh oh!

Conversation

hubgeter commented Nov 27, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

hubgeter commented Dec 1, 2024

Uh oh!

doris-robot commented Dec 1, 2024

Uh oh!

doris-robot commented Dec 1, 2024

Uh oh!

doris-robot commented Dec 1, 2024

Uh oh!

hubgeter commented Dec 3, 2024

Uh oh!

doris-robot commented Dec 3, 2024

Uh oh!

doris-robot commented Dec 3, 2024

Uh oh!

doris-robot commented Dec 3, 2024

Uh oh!

github-actionsBot commented Dec 4, 2024

Uh oh!

github-actionsBot commented Dec 4, 2024

Uh oh!

hubgeter commented Dec 5, 2024

Uh oh!

hubgeter commented Dec 5, 2024

Uh oh!

hubgeter commented Dec 5, 2024

Uh oh!

doris-robot commented Dec 5, 2024

Uh oh!

doris-robot commented Dec 5, 2024

Uh oh!

doris-robot commented Dec 5, 2024

Uh oh!

github-actionsBot commented Dec 5, 2024

Uh oh!

suxiaogang223 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hubgeter commented Nov 27, 2024•
edited
Loading