branch-4.1: [feature](score) support BM25 scoring in inverted index query_v2 #59847 by airborne12 · Pull Request #61472 · apache/doris

airborne12 · 2026-03-18T08:03:35Z

Proposed changes

Cherry-pick #59847 to branch-4.1.

Original PR: #59847

Further comments

Resolved cherry-pick conflicts (minor differences from branch-4.0 pick):

Added IndexQueryContextPtr member and set/get_index_query_context to IndexExecContext
Extended evaluate_inverted_index_with_search_param with index_query_context parameter
Added enable_inverted_index_wand_query to thrift and SessionVariable (field 203)
Kept existing branch-4.1 fields (single_backend_query field 202) intact

hello-stephen · 2026-03-18T08:03:41Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

airborne12 · 2026-03-18T08:04:15Z

run buildall

airborne12 · 2026-03-18T08:16:15Z

run buildall

doris-robot · 2026-03-18T08:40:53Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.15% (1788/2259)
Line Coverage	64.42% (31942/49583)
Region Coverage	65.22% (15978/24499)
Branch Coverage	55.80% (8502/15236)

…che#59847)

…M25 scoring 1. Update contrib/clucene from 8b57674 to c51b5cc to include: - ac9475a: block max WAND algorithm with BM25 similarity - aef5c9c: Fix GCC -Werror compilation errors - c51b5cc: Fix GCC -Werror=overloaded-virtual in FieldForMerge Required for readBlock, getMaxBlockFreq, getMaxBlockNorm, getLastDocInBlock APIs used by segment_postings.h 2. Fix doc_set_collector: When context.readers is empty (e.g., AllQuery for MATCH_ALL_DOCS), for_each_index_segment returns immediately without iterating. Added fallback to create scorer directly from weight, which allows AllScorer to work without an IndexReader.

airborne12 · 2026-03-19T06:54:11Z

run buildall

hello-stephen · 2026-03-19T07:30:43Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.06% (1786/2259)
Line Coverage	64.39% (31927/49583)
Region Coverage	65.24% (15982/24499)
Branch Coverage	55.83% (8506/15236)

…undle

airborne12 · 2026-03-19T08:07:07Z

run buildall

hello-stephen · 2026-03-19T08:32:22Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	78.54% (1786/2274)
Line Coverage	64.31% (31924/49641)
Region Coverage	65.14% (15979/24532)
Branch Coverage	55.72% (8502/15258)

The branch-4.0 stub (static, returns false) conflicts with the real implementation in the anonymous namespace added by the BM25 scoring PR. On branch-4.1 the real implementation is correct, so remove the stub to fix the ambiguous call compilation error.

airborne12 · 2026-03-19T08:42:11Z

run buildall

doris-robot · 2026-03-19T09:04:45Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	78.54% (1786/2274)
Line Coverage	64.31% (31924/49641)
Region Coverage	65.11% (15972/24532)
Branch Coverage	55.72% (8502/15258)

hello-stephen · 2026-03-19T09:50:44Z

FE UT Coverage Report

Increment line coverage 33.33% (2/6) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2026-03-19T11:12:05Z

BE UT Coverage Report

Increment line coverage 24.63% (183/743) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.77% (19412/36787)
Line Coverage	36.20% (181479/501347)
Region Coverage	32.65% (140163/429298)
Branch Coverage	33.65% (61035/181393)

hello-stephen · 2026-03-19T11:45:52Z

BE Regression && UT Coverage Report

Increment line coverage 33.56% (251/748) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.35% (25690/36007)
Line Coverage	54.24% (271423/500406)
Region Coverage	51.48% (223169/433535)
Branch Coverage	53.07% (96594/182019)

airborne12 requested a review from yiguolei as a code owner March 18, 2026 08:03

airborne12 force-pushed the pick/branch-4.1/59847 branch from 4a5d6d0 to 094e120Compare March 18, 2026 08:11

zzzxl1993and others added 2 commits March 19, 2026 14:51

[feature](score) support BM25 scoring in inverted index query_v2 (apa…
e82c418
…che#59847)

airborne12 force-pushed the pick/branch-4.1/59847 branch from 46d977d to 6cb69d6Compare March 19, 2026 06:51

[fix](score) Remove duplicate field_name_to_column_id in SearchInputB…
1dd3483
…undle

yiguolei approved these changes Mar 19, 2026
View reviewed changes

yiguolei merged commit 75bb9bd into apache:branch-4.1Mar 19, 2026
24 of 28 checks passed

Uh oh!

Conversation

airborne12 commented Mar 18, 2026

Proposed changes

Further comments

Uh oh!

hello-stephen commented Mar 18, 2026

Uh oh!

airborne12 commented Mar 18, 2026

Uh oh!

airborne12 commented Mar 18, 2026

Uh oh!

doris-robot commented Mar 18, 2026

Cloud UT Coverage Report

Uh oh!

airborne12 commented Mar 19, 2026

Uh oh!

hello-stephen commented Mar 19, 2026

Cloud UT Coverage Report

Uh oh!

airborne12 commented Mar 19, 2026

Uh oh!

hello-stephen commented Mar 19, 2026

Cloud UT Coverage Report

Uh oh!

airborne12 commented Mar 19, 2026

Uh oh!

doris-robot commented Mar 19, 2026

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Mar 19, 2026

FE UT Coverage Report

Uh oh!

doris-robot commented Mar 19, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Mar 19, 2026

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants