[refactor](storage) replace TypeInfo polymorphic hierarchy with FieldType by csun5285 · Pull Request #63153 · apache/doris

csun5285 · 2026-05-12T02:29:17Z

Replace TypeInfo* fields and parameters with FieldType throughout the storage layer, and add a static field_type_size(FieldType) helper that maps each type to sizeof(CppTypeTraits::CppType) so size lookups stay in sync with the existing traits machinery.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-05-12T02:29:23Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

csun5285 · 2026-05-12T02:29:57Z

run buildall

hello-stephen · 2026-05-12T04:01:48Z

TPC-H: Total hot run time: 29787 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f81e5db6b2ba28e55ccbfa3a6b0ee0a82a342eb7, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17626	4005	3955	3955
q2	q3	10730	907	635	635
q4	4692	456	348	348
q5	7573	1327	1133	1133
q6	258	171	138	138
q7	946	977	752	752
q8	10092	1429	1291	1291
q9	6292	5406	5313	5313
q10	6332	2086	1787	1787
q11	480	266	255	255
q12	684	422	289	289
q13	18277	3340	2778	2778
q14	292	282	266	266
q15	q16	921	845	783	783
q17	1027	934	859	859
q18	6524	5682	5647	5647
q19	1351	1292	1118	1118
q20	517	404	268	268
q21	5204	2296	1867	1867
q22	422	351	305	305
Total cold run time: 100240 ms
Total hot run time: 29787 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4221	4137	4133	4133
q2	q3	4623	4764	4173	4173
q4	2103	2182	1401	1401
q5	4978	4968	5279	4968
q6	192	166	128	128
q7	2264	1837	1700	1700
q8	3522	3153	3172	3153
q9	8522	8534	8372	8372
q10	4485	4495	4315	4315
q11	617	431	415	415
q12	696	777	517	517
q13	3247	3629	2892	2892
q14	302	308	287	287
q15	q16	782	806	712	712
q17	1376	1315	1460	1315
q18	8074	7028	7002	7002
q19	1168	1171	1190	1171
q20	2241	2278	1963	1963
q21	6178	5571	4859	4859
q22	574	501	431	431
Total cold run time: 60165 ms
Total hot run time: 53907 ms

hello-stephen · 2026-05-12T04:12:45Z

TPC-DS: Total hot run time: 170297 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f81e5db6b2ba28e55ccbfa3a6b0ee0a82a342eb7, data reload: false
query5	4302	647	523	523
query6	329	226	195	195
query7	4241	601	302	302
query8	318	231	219	219
query9	8812	4038	4050	4038
query10	464	338	305	305
query11	5779	2396	2192	2192
query12	178	132	130	130
query13	1275	601	419	419
query14	6442	5351	5057	5057
query14_1	4363	4383	4317	4317
query15	215	208	186	186
query16	1015	492	468	468
query17	1214	789	652	652
query18	2738	506	366	366
query19	260	207	169	169
query20	136	135	133	133
query21	215	136	119	119
query22	13705	13778	14521	13778
query23	17412	16653	16303	16303
query23_1	16314	16255	16294	16255
query24	8011	1829	1397	1397
query24_1	1378	1354	1337	1337
query25	560	482	430	430
query26	1329	310	201	201
query27	2628	607	342	342
query28	4306	1946	1931	1931
query29	996	626	513	513
query30	294	232	198	198
query31	1114	1064	926	926
query32	82	73	69	69
query33	543	340	293	293
query34	1152	1150	653	653
query35	779	788	669	669
query36	1364	1324	1178	1178
query37	147	98	88	88
query38	3194	3129	3093	3093
query39	922	931	894	894
query39_1	868	873	869	869
query40	239	163	135	135
query41	63	60	61	60
query42	112	111	108	108
query43	319	324	283	283
query44 query45	214	228	190	190
query46	1061	1199	763	763
query47	2381	2288	2232	2232
query48	417	406	282	282
query49	632	530	419	419
query50	721	280	211	211
query51	4344	4248	4347	4248
query52	105	105	93	93
query53	246	271	201	201
query54	315	266	254	254
query55	100	89	83	83
query56	289	291	301	291
query57	1429	1399	1327	1327
query58	290	274	268	268
query59	1562	1644	1397	1397
query60	348	327	327	327
query61	184	149	161	149
query62	680	618	563	563
query63	238	206	202	202
query64	2349	822	685	685
query65 query66	1689	503	397	397
query67	29908	29198	29750	29198
query68 query69	468	332	299	299
query70	1057	1003	1011	1003
query71	308	275	266	266
query72	2992	2676	2590	2590
query73	842	790	406	406
query74	5084	4871	4702	4702
query75	2768	2661	2331	2331
query76	2299	1164	761	761
query77	427	423	355	355
query78	12945	12989	12322	12322
query79	1522	1016	724	724
query80	1421	604	469	469
query81	523	282	243	243
query82	877	162	118	118
query83	344	265	245	245
query84	261	141	121	121
query85	895	545	464	464
query86	468	345	312	312
query87	3412	3330	3195	3195
query88	3578	2704	2661	2661
query89	439	388	336	336
query90	1976	179	178	178
query91	180	170	136	136
query92	78	77	71	71
query93	1118	946	541	541
query94	712	336	258	258
query95	673	469	355	355
query96	1108	796	353	353
query97	2710	2694	2583	2583
query98	238	228	226	226
query99	1113	1112	969	969
Total cold run time: 255371 ms
Total hot run time: 170297 ms

zclllyybb · 2026-05-12T04:46:47Z

/review

github-actions

This refactor is not ready to merge because it removes the storage TypeInfo helper API while leaving multiple BE test files that still call the removed symbols, so those test targets will fail to compile.

Critical checkpoint conclusions:

Goal/test proof: The intended storage refactor is only partially applied; existing tests are not fully migrated, and the PR marks no tests as run.
Scope/focus: The production changes are focused, but the migration is incomplete across test code.
Concurrency/lifecycle/config/compatibility: No new concurrency, special lifecycle, config, persistence, or wire/storage format compatibility concerns were identified in the reviewed changes.
Parallel paths: Parallel BE storage test paths still use the removed API and must be updated.
Test coverage/results: Existing tests were modified in a few files, but many affected tests remain unmigrated; no generated results are involved.
Observability/performance: No additional observability or performance issue found for this refactor.
User focus: No additional user-provided review focus was specified.

…Type The TypeInfo class hierarchy (TypeInfo / ScalarTypeInfo / ArrayTypeInfo / MapTypeInfo / StructTypeInfo) only exposed two operations -- type() and size() -- both fully derivable from FieldType. Storing a TypeInfoPtr alongside the TabletColumn / Field that already carries the same information was redundant. Replace TypeInfo* fields and parameters with FieldType throughout the storage layer, and add a static field_type_size(FieldType) helper that maps each type to sizeof(CppTypeTraits<FT>::CppType) so size lookups stay in sync with the existing traits machinery. Index writer's ARRAY peel-off now reads the inner element type from TabletColumn::get_sub_column(0).type() instead of dynamic_cast'ing to ArrayTypeInfo, which removes the only consumer of nested TypeInfo accessors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

csun5285 · 2026-05-12T09:25:33Z

run buildall

github-actions · 2026-05-12T09:46:53Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2026-05-12T11:07:21Z

TPC-H: Total hot run time: 29627 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2ebab5090b54f4eec5fea820e9dc4f386dacd30, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17650	3887	3861	3861
q2	q3	10699	1031	616	616
q4	4660	471	345	345
q5	7456	1362	1144	1144
q6	187	168	138	138
q7	909	980	741	741
q8	9304	1446	1281	1281
q9	5538	5450	5397	5397
q10	6266	2078	1823	1823
q11	469	265	260	260
q12	628	412	289	289
q13	18088	3360	2738	2738
q14	297	284	260	260
q15	q16	906	880	788	788
q17	994	1084	697	697
q18	6500	5707	5534	5534
q19	1326	1162	1148	1148
q20	508	398	262	262
q21	5008	2430	1953	1953
q22	464	422	352	352
Total cold run time: 97857 ms
Total hot run time: 29627 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4795	4766	4953	4766
q2	q3	4760	4779	4180	4180
q4	2129	2205	1412	1412
q5	5049	5025	5308	5025
q6	200	184	130	130
q7	2075	1783	1591	1591
q8	3406	3107	3131	3107
q9	8473	8596	8479	8479
q10	4518	4486	4246	4246
q11	594	434	401	401
q12	715	761	509	509
q13	3220	3642	2933	2933
q14	394	350	292	292
q15	q16	815	800	694	694
q17	1340	1304	1262	1262
q18	7975	7265	7183	7183
q19	1161	1155	1178	1155
q20	2238	2228	1979	1979
q21	6231	5452	4903	4903
q22	556	515	411	411
Total cold run time: 60644 ms
Total hot run time: 54658 ms

hello-stephen · 2026-05-12T11:18:15Z

TPC-DS: Total hot run time: 171000 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2ebab5090b54f4eec5fea820e9dc4f386dacd30, data reload: false
query5	4326	663	528	528
query6	336	224	209	209
query7	4218	556	295	295
query8	332	231	227	227
query9	8813	4025	4031	4025
query10	446	335	299	299
query11	5784	2368	2176	2176
query12	178	129	124	124
query13	1292	587	423	423
query14	6012	5373	5074	5074
query14_1	4381	4409	4319	4319
query15	213	203	179	179
query16	991	469	447	447
query17	922	755	610	610
query18	2448	474	344	344
query19	218	201	163	163
query20	135	133	127	127
query21	214	138	115	115
query22	13613	13634	13381	13381
query23	17114	16337	15968	15968
query23_1	16033	16244	16178	16178
query24	7395	1761	1363	1363
query24_1	1359	1346	1378	1346
query25	554	500	441	441
query26	1305	308	178	178
query27	2715	597	335	335
query28	4451	1984	1932	1932
query29	1046	630	513	513
query30	312	237	205	205
query31	1132	1056	937	937
query32	86	77	72	72
query33	540	356	284	284
query34	1163	1125	651	651
query35	750	783	670	670
query36	1338	1312	1188	1188
query37	152	102	89	89
query38	3188	3141	3020	3020
query39	933	931	897	897
query39_1	888	895	883	883
query40	237	156	137	137
query41	82	76	62	62
query42	111	107	106	106
query43	323	333	291	291
query44 query45	211	203	193	193
query46	1092	1152	727	727
query47	2328	2335	2178	2178
query48	401	436	284	284
query49	638	544	427	427
query50	704	286	224	224
query51	4356	4233	4204	4204
query52	110	108	94	94
query53	250	280	208	208
query54	305	271	259	259
query55	92	88	84	84
query56	293	300	303	300
query57	1411	1415	1337	1337
query58	319	271	271	271
query59	1556	1611	1419	1419
query60	360	341	360	341
query61	156	152	155	152
query62	681	623	561	561
query63	249	200	208	200
query64	2425	832	673	673
query65 query66	1755	511	398	398
query67	30082	30006	29869	29869
query68 query69	461	332	312	312
query70	1038	993	993	993
query71	298	273	268	268
query72	2930	2773	2601	2601
query73	859	767	432	432
query74	5075	4891	4759	4759
query75	2786	2740	2347	2347
query76	2294	1130	738	738
query77	431	435	360	360
query78	12933	12867	12388	12388
query79	1459	940	757	757
query80	702	649	531	531
query81	462	287	247	247
query82	848	170	126	126
query83	376	298	272	272
query84	324	146	118	118
query85	964	587	561	561
query86	385	331	344	331
query87	3410	3344	3223	3223
query88	3503	2694	2650	2650
query89	432	386	336	336
query90	1913	186	182	182
query91	179	166	142	142
query92	81	78	74	74
query93	938	952	540	540
query94	534	345	294	294
query95	653	383	444	383
query96	1107	754	318	318
query97	2704	2729	2576	2576
query98	239	235	229	229
query99	1124	1113	1005	1005
Total cold run time: 252336 ms
Total hot run time: 171000 ms

hello-stephen · 2026-05-12T15:07:41Z

BE Regression && UT Coverage Report

Increment line coverage 84.96% (113/133) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	63.52% (23938/37683)
Line Coverage	47.15% (246336/522444)
Region Coverage	44.14% (202228/458113)
Branch Coverage	45.43% (87605/192815)

hello-stephen · 2026-05-13T02:16:15Z

/review

github-actions

Automated review completed for b2ebab5.

I did not find additional distinct issues beyond the inline threads that are already open, so I am not duplicating those comments.

Critical checkpoint conclusions:

Goal/test evidence: the PR consistently replaces BE storage TypeInfo plumbing with FieldType/field_type_size in the modified storage/index paths. Existing CI shows Doris compile and BE UT TeamCity checks passing; the GitHub macOS BE UT failure is an environment JDK-25-vs-JDK-17 setup failure.
Scope: the change is broad but mechanically focused on removing TypeInfo from storage/index readers/writers and associated tests.
Concurrency/lifecycle: no new shared mutable state, locks, thread entries, or cross-TU static dependencies were introduced in the reviewed diff.
Config/compatibility: no new config items or persisted format changes were introduced; existing serialized field type and encoding metadata remain unchanged.
Parallel paths: the reviewed primary key, bloom filter, zone map, inverted index, column reader/writer, and default-value paths were updated consistently in the touched code.
Error handling: status propagation is preserved in the reviewed paths; unsupported type handling remains present where readers/writers initialize. Existing open comments already cover questions around NONE/UNKNOWN handling and variable-length type sizing.
Data correctness/version/delete bitmap: the change does not modify visible-version selection, rowset lifecycle, transaction persistence, or MoW delete-bitmap semantics.
Memory/performance: no new large allocations or memory-tracking-sensitive containers were introduced; the change removes TypeInfo allocation/indirection.
Test coverage: touched unit tests were migrated in part, and existing review context already discusses remaining TypeInfo test references, so I did not repeat that thread.

User focus: no additional user-provided focus points were present.

dismiss

github-actions · 2026-05-13T02:35:12Z

PR approved by anyone and no changes requested.

airborne12

LGTM

…Type (apache#63153) Replace TypeInfo* fields and parameters with FieldType throughout the storage layer, and add a static field_type_size(FieldType) helper that maps each type to sizeof(CppTypeTraits<FT>::CppType) so size lookups stay in sync with the existing traits machinery.

…Type (apache#63153) Replace TypeInfo* fields and parameters with FieldType throughout the storage layer, and add a static field_type_size(FieldType) helper that maps each type to sizeof(CppTypeTraits<FT>::CppType) so size lookups stay in sync with the existing traits machinery. (cherry picked from commit 7e5c941)

…anch-4.1 (#63153 #63233 #63622 #63291 #64341) (#64550) pick from master #63153 #63233 #63622 #63291 #64341 --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

csun5285 requested a review from airborne12 as a code owner May 12, 2026 02:29

github-actionsBot previously requested changes May 12, 2026
View reviewed changes

Comment threadbe/src/storage/types.h

csun5285 force-pushed the refactor/replace-typeinfo-with-fieldtype branch from f81e5db to b2ebab5Compare May 12, 2026 09:25

Mryange approved these changes May 12, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 12, 2026