[refine](function) use typed ANN query vector by Mryange · Pull Request #63834 · apache/doris

Mryange · 2026-05-28T09:34:47Z

What problem does this PR solve?

Problem Summary:
ANN query vector extraction returned a generic IColumn::Ptr, so the TopN and range search paths had to downcast the column again before reading float data. This made the code more indirect and delayed type validation. This PR changes the helper and runtime state to keep the query vector as ColumnFloat32::Ptr, validates the concrete type at extraction time, and removes redundant casts from the ANN execution path.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-05-28T09:34:53Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Mryange · 2026-05-28T09:35:11Z

/review

github-actions

Review result: no blocking findings in the actual PR diff. The change is limited to keeping ANN query vectors as ColumnFloat32::Ptr after extraction, removing downstream casts, and adding a negative unit test for non-Float32 vector data.

Critical checkpoint conclusions:

Goal/test: The code accomplishes the stated goal of validating/exposing the ANN query vector as Float32 at extraction time. Existing ANN tests are updated and a NonFloatArrayFails unit test covers the new validation path.
Scope/minimality: The GitHub PR diff is small and focused on ANN runtime/query-vector typing.
Concurrency/lifecycle: No new concurrency path is introduced. The shared column pointer lifetime remains owned by Doris column COW pointers; range/topn runtime copies continue to share the immutable constant column data.
Configuration/compatibility: No new configuration, persisted format, thrift protocol, or storage compatibility change is introduced.
Parallel paths: Both TopN and range-search ANN paths are updated to use ColumnFloat32::Ptr.
Error handling: Non-Float32 query vector extraction now returns InvalidArgument instead of relying on a later cast; existing Status propagation is preserved.
Memory/performance: No significant new allocation or hot-path overhead; the change removes repeated casts.
Observability: No additional observability appears necessary for this refactor.
User focus: No additional user-provided focus points were present.

I did not run tests in this review environment.

Mryange · 2026-05-28T11:36:26Z

run buildall

hello-stephen · 2026-05-28T13:12:38Z

TPC-H: Total hot run time: 31411 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 258b82887f34c0b5206b7aa1837106390f25e565, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17649	3947	3915	3915
q2	q3	10845	1413	819	819
q4	4700	486	352	352
q5	7685	2325	2160	2160
q6	249	179	135	135
q7	1015	786	637	637
q8	9404	1791	1695	1695
q9	5290	4989	4954	4954
q10	6417	2208	1889	1889
q11	445	288	245	245
q12	641	425	293	293
q13	18153	3428	2787	2787
q14	265	259	236	236
q15	q16	823	773	712	712
q17	933	884	1000	884
q18	7260	5866	5579	5579
q19	1302	1264	1094	1094
q20	528	413	258	258
q21	5962	2602	2463	2463
q22	448	370	304	304
Total cold run time: 100014 ms
Total hot run time: 31411 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4312	4233	4245	4233
q2	q3	4534	4951	4344	4344
q4	2073	2202	1383	1383
q5	4437	4308	4393	4308
q6	363	214	160	160
q7	2262	1956	1665	1665
q8	2544	2231	2478	2231
q9	8335	7974	8137	7974
q10	4805	4761	4353	4353
q11	601	418	366	366
q12	766	749	554	554
q13	3223	3719	2995	2995
q14	302	313	273	273
q15	q16	713	727	630	630
q17	1319	1372	1405	1372
q18	8294	7347	7277	7277
q19	1174	1095	1079	1079
q20	2216	2223	1942	1942
q21	5259	4577	4411	4411
q22	521	449	399	399
Total cold run time: 58053 ms
Total hot run time: 51949 ms

hello-stephen · 2026-05-28T13:23:38Z

TPC-DS: Total hot run time: 172299 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 258b82887f34c0b5206b7aa1837106390f25e565, data reload: false
query5	4331	669	531	531
query6	332	222	197	197
query7	4230	560	308	308
query8	330	232	215	215
query9	8821	4116	4091	4091
query10	446	347	301	301
query11	5807	2396	2265	2265
query12	178	128	123	123
query13	1313	602	419	419
query14	6095	5480	5193	5193
query14_1	4592	4481	4465	4465
query15	217	205	188	188
query16	1017	473	403	403
query17	1116	722	590	590
query18	2715	474	346	346
query19	235	202	168	168
query20	132	129	127	127
query21	218	149	116	116
query22	13681	13592	13346	13346
query23	17491	16585	16238	16238
query23_1	16466	16333	16360	16333
query24	7492	1767	1341	1341
query24_1	1329	1325	1359	1325
query25	562	465	422	422
query26	1325	316	169	169
query27	2684	538	346	346
query28	4411	2020	2008	2008
query29	956	643	486	486
query30	311	243	197	197
query31	1134	1082	962	962
query32	86	74	73	73
query33	549	369	294	294
query34	1180	1135	654	654
query35	781	815	710	710
query36	1398	1439	1257	1257
query37	153	110	88	88
query38	3239	3199	3153	3153
query39	939	916	901	901
query39_1	875	882	890	882
query40	229	152	127	127
query41	74	68	67	67
query42	113	110	111	110
query43	350	343	301	301
query44 query45	216	213	210	210
query46	1088	1230	748	748
query47	2374	2393	2222	2222
query48	410	425	304	304
query49	652	509	415	415
query50	1015	351	259	259
query51	4407	4420	4283	4283
query52	108	110	99	99
query53	254	289	212	212
query54	328	311	270	270
query55	98	94	94	94
query56	319	324	335	324
query57	1450	1434	1341	1341
query58	311	293	272	272
query59	1624	1721	1491	1491
query60	332	340	319	319
query61	185	176	182	176
query62	712	685	586	586
query63	251	204	211	204
query64	2442	854	698	698
query65 query66	1674	493	385	385
query67	29190	29701	29665	29665
query68 query69	475	346	319	319
query70	1050	1044	1017	1017
query71	314	279	278	278
query72	3027	2688	2345	2345
query73	889	815	428	428
query74	5148	4947	4769	4769
query75	2682	2594	2249	2249
query76	2280	1172	790	790
query77	415	432	351	351
query78	12454	12303	11848	11848
query79	1726	1062	756	756
query80	728	551	454	454
query81	471	281	246	246
query82	1408	198	127	127
query83	326	280	248	248
query84	271	146	111	111
query85	906	534	459	459
query86	432	333	330	330
query87	3461	3408	3254	3254
query88	3666	2753	2766	2753
query89	455	392	342	342
query90	1845	179	175	175
query91	174	168	148	148
query92	81	79	75	75
query93	1608	1499	934	934
query94	611	358	334	334
query95	690	460	350	350
query96	998	778	351	351
query97	2699	2751	2598	2598
query98	232	231	230	230
query99	1152	1181	1038	1038
Total cold run time: 255030 ms
Total hot run time: 172299 ms

Mryange · 2026-05-29T00:48:04Z

run beut

hello-stephen · 2026-05-29T02:11:24Z

BE UT Coverage Report

Increment line coverage 100.00% (12/12) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.99% (20979/38860)
Line Coverage	37.54% (198892/529809)
Region Coverage	33.81% (155834/460849)
Branch Coverage	34.83% (67889/194917)

hello-stephen · 2026-05-29T02:35:46Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (12/12) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.82% (28092/38054)
Line Coverage	57.76% (305192/528422)
Region Coverage	54.95% (255666/465240)
Branch Coverage	56.44% (110404/195623)

ANN query vector extraction returned a generic `IColumn::Ptr`, so the TopN and range search paths had to downcast the column again before reading float data. This made the code more indirect and delayed type validation. This PR changes the helper and runtime state to keep the query vector as `ColumnFloat32::Ptr`, validates the concrete type at extraction time, and removes redundant casts from the ANN execution path.

This reverts commit a6f53a1.

upd
258b828

github-actionsBot reviewed May 28, 2026
View reviewed changes

zclllyybb approved these changes May 29, 2026
View reviewed changes

zclllyybb merged commit 99691f6 into apache:masterMay 29, 2026
31 of 32 checks passed

Mryange mentioned this pull request Jun 15, 2026
[fix](be) Backport selected query engine fixes to branch-4.1 #64537
Merged
16 tasks

Mryange added a commit to Mryange/doris that referenced this pull request Jun 16, 2026
Revert "[refine](function) use typed ANN query vector (apache#63834)"
35e28b6
This reverts commit a6f53a1.

Navigation Menu

Uh oh!

[refine](function) use typed ANN query vector - #63834

[refine](function) use typed ANN query vector#63834
zclllyybb merged 1 commit into
apache:masterfrom
Mryange:use-typed-ANN-query-vector

Mryange commented May 28, 2026

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

Mryange commented May 28, 2026

Uh oh!

github-actionsBot left a comment

Uh oh!

Mryange commented May 28, 2026

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

Mryange commented May 29, 2026

Uh oh!

hello-stephen commented May 29, 2026

Uh oh!

hello-stephen commented May 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Mryange commented May 28, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

Mryange commented May 28, 2026

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Mryange commented May 28, 2026

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

hello-stephen commented May 28, 2026

Uh oh!

Mryange commented May 29, 2026

Uh oh!

hello-stephen commented May 29, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented May 29, 2026

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants