[fix](be) Normalize SNII prefix syntax before analysis

[airborne12]

What problem does this PR solve?

Issue Number: None

Related PR: #66052

Problem Summary:

SEARCH ... PREFIX values retain their trailing DSL * marker in FE. SNII classifies a custom keyword analyzer as analyzed, but that analyzer preserves the marker as a literal byte. Consequently, a predicate such as fail* sent fail* to the phrase-prefix reader instead of the indexed prefix fail and returned no rows, while V3 matched the expected documents.

This PR strips exactly the trailing DSL marker before SNII phrase-prefix analysis. Non-analyzed keyword fields continue to use WILDCARD_QUERY with the marker intact, ordinary WILDCARD clauses are unchanged, and the V3 path is unchanged. The change is query-only: it does not alter index writes or the on-disk format, so existing SNII indexes are fixed after upgrade without rebuilding.

TDD evidence:

• RED: a real custom keyword tokenizer/analyzer preserved fail*; FunctionSearch::build_leaf_query sent fail* and produced an empty bitmap instead of {0, 2}.
• GREEN: the same path sends fail, retains MATCH_PHRASE_PREFIX_QUERY, and produces {0, 2}.
• All 98 FunctionSearchTest cases pass.
• ./build.sh --be -j 192 passes with ASAN.
• build-support/check-format.sh passes.

Release note

Fix SEARCH PREFIX queries on SNII indexes that use a custom keyword analyzer.

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes. SNII phrase-prefix queries no longer pass the PREFIX DSL * syntax marker to analyzed readers.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[airborne12]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 17055 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 269027914bcd374b2e7500f36b516e781d0fa4df, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17562	3140	3073	3073
q2	q3	10859	855	499	499
q4	4675	261	214	214
q5	7653	565	389	389
q6	139	115	97	97
q7	526	495	388	388
q8	9249	860	904	860
q9	3423	2366	2408	2366
q10	6504	860	711	711
q11	439	264	247	247
q12	685	406	331	331
q13	17882	1556	1161	1161
q14	161	147	142	142
q15	q16	448	398	370	370
q17	774	735	809	735
q18	3072	2280	2266	2266
q19	1114	849	791	791
q20	694	531	489	489
q21	5239	1693	1891	1693
q22	326	269	233	233
Total cold run time: 91424 ms
Total hot run time: 17055 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	3519	3417	3440	3417
q2	q3	2208	2260	2127	2127
q4	1196	1163	892	892
q5	2205	2131	2138	2131
q6	171	117	87	87
q7	1064	913	874	874
q8	1619	1436	1430	1430
q9	3169	3125	3126	3125
q10	1900	1790	1620	1620
q11	367	284	261	261
q12	458	423	338	338
q13	1496	1532	1194	1194
q14	174	176	177	176
q15	q16	406	389	363	363
q17	1066	1043	1043	1043
q18	4922	4427	4810	4427
q19	879	835	836	835
q20	979	927	802	802
q21	3577	3291	3272	3272
q22	423	352	336	336
Total cold run time: 31798 ms
Total hot run time: 28750 ms

[hello-stephen]

TPC-DS: Total hot run time: 81239 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 269027914bcd374b2e7500f36b516e781d0fa4df, data reload: false
query5	4280	436	350	350
query6	428	163	155	155
query7	4809	462	252	252
query8	290	127	112	112
query9	8680	2926	2930	2926
query10	407	248	219	219
query11	5363	1046	911	911
query12	111	71	70	70
query13	1193	439	314	314
query14	6020	2238	2118	2118
query14_1	2052	1987	1973	1973
query15	170	122	113	113
query16	925	381	357	357
query17	776	445	352	352
query18	2311	320	226	226
query19	159	144	109	109
query20	68	68	71	68
query21	210	116	100	100
query22	5341	5255	5244	5244
query23	6536	6259	5969	5969
query23_1	6174	6120	5968	5968
query24	7294	1128	765	765
query24_1	779	771	766	766
query25	411	291	237	237
query26	1251	252	168	168
query27	2730	420	297	297
query28	4631	1491	1499	1491
query29	928	418	336	336
query30	268	171	156	156
query31	835	427	356	356
query32	97	48	46	46
query33	454	206	169	169
query34	981	831	500	500
query35	399	399	334	334
query36	559	556	511	511
query37	117	81	70	70
query38	993	861	828	828
query39	500	490	459	459
query39_1	468	469	470	469
query40	217	126	111	111
query41	52	52	54	52
query42	85	78	76	76
query43	240	244	213	213
query44 query45	106	105	98	98
query46	756	839	537	537
query47	780	763	719	719
query48	304	310	235	235
query49	544	237	181	181
query50	802	344	258	258
query51	8283	8200	8214	8200
query52	74	74	68	68
query53	220	225	166	166
query54	265	177	171	171
query55	72	59	58	58
query56	269	237	258	237
query57	710	648	652	648
query58	225	202	197	197
query59	1239	1228	1122	1122
query60	270	200	189	189
query61	131	128	123	123
query62	348	213	201	201
query63	194	159	161	159
query64	2703	719	604	604
query65 query66	1918	299	249	249
query67	10498	9989	9880	9880
query68 query69	425	228	202	202
query70	633	638	629	629
query71	313	258	248	248
query72	2372	1755	1629	1629
query73	683	603	366	366
query74	1998	1242	1154	1154
query75	1252	1171	1042	1042
query76	2371	718	531	531
query77	260	249	216	216
query78	4004	3745	3397	3397
query79	1257	798	580	580
query80	1182	401	351	351
query81	488	210	175	175
query82	629	143	119	119
query83	358	271	238	238
query84 query85	902	511	441	441
query86	387	174	165	165
query87	1015	1002	927	927
query88	2846	2184	2200	2184
query89	321	241	222	222
query90	1948	159	174	159
query91	168	145	127	127
query92	51	49	46	46
query93	1485	1163	817	817
query94	664	277	222	222
query95	610	352	407	352
query96	829	587	289	289
query97	1136	1106	1046	1046
query98	145	141	139	139
query99	428	365	315	315
Total cold run time: 171855 ms
Total hot run time: 81239 ms

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.74% (34093/45016)
Line Coverage	60.70% (384053/632666)
Region Coverage	56.88% (322352/566736)
Branch Coverage	57.68% (146874/254614)

[airborne12]

/review

[github-actions]

Automated code review summary

Overall conclusion: no blocking correctness issue or other substantiated review finding was identified. The change is focused and the reviewed behavior is consistent across the relevant SEARCH and inverted-index paths.

Critical checkpoints:

• Goal and proof: the patch removes exactly the trailing PREFIX DSL syntax marker before analyzed SNII phrase-prefix processing. The added unit test uses a real custom keyword tokenizer/analyzer to prove that fail* otherwise remains a literal token, then verifies that build_leaf_query sends fail with MATCH_PHRASE_PREFIX_QUERY and returns the expected bitmap. The existing non-analyzed keyword PREFIX test continues to prove the WILDCARD route keeps *.
• Scope and clarity: only the SNII-native value normalization/logging and its targeted FunctionSearch test change. The implementation reuses the existing query-type split and keeps the V3/CLucene and direct-reader paths untouched.
• Concurrency and lifecycle: production changes use request-local state and add no locks or shared mutable state. The test-scoped IndexPolicyMgr is restored with DEFER before the stack manager is destroyed, matching the established test pattern; policy cleanup is explicit.
• Conditions and parallel paths: FE standard and Lucene parser modes are the only production PREFIX producers, standalone * becomes MATCH_ALL_DOCS, thrift serialization preserves the suffix, and escaped literal-star forms retain literal data when the single syntax byte is removed. Non-analyzed PREFIX is rerouted to WILDCARD before normalization; ordinary WILDCARD, null-bitmap handling, and V3 behavior remain unchanged.
• Analyzer, cache, and execution semantics: the stripped logical prefix is used consistently for SNII analysis, raw-query/cache and single-flight identity, CommonGrams planning, prefix expansion and max-expansions handling, and optional scoring. No analyzer needs the DSL marker as indexed content.
• Compatibility and persistence: this is query-only. It adds no configuration, FE-BE field, symbol/storage-format change, write path, transaction state, or persisted metadata, so no index rebuild or replay/rolling-format compatibility path is required.
• Error handling and observability: existing Status propagation is preserved; the new DORIS_CHECKs encode the verified FE/serialization invariant. Debug logging retains the original caller value whenever normalization changes it.
• Performance: the existing per-leaf string copy remains, with only a constant-time suffix check and removal; no hot-path scan, allocation class, or I/O behavior is added.
• Test and CI evidence: targeted BE unit coverage is present. Relevant live checks for this head are green, including BE UT, macOS BE UT, compile, clang formatting, P0/nonconcurrent/external regression, dependency review, and coverage. Per the runner review instructions, no local build was run during this review.
• User focus: no additional user-provided focus was specified.

Review convergence: two complementary full-review agents and one dedicated PREFIX/analyzer risk agent all returned NO_NEW_VALUABLE_FINDINGS in round 1. The main-agent final changed-file and unresolved-candidate sweep also found no remaining suspicious point. No inline comments are proposed.

[eldenmoon]

LGTM