Uh oh!
There was an error while loading. Please reload this page.
[fix](search) Fix wildcard query on variant subcolumns returning empty results - #60793
Conversation
…y results When search() DSL uses wildcard patterns (e.g. *ith, sm*th) on variant subcolumns with analyzer-based indexes, the FieldReaderResolver only upgraded EQUAL_QUERY to MATCH_ANY_QUERY but not WILDCARD_QUERY. This caused select_best_reader to pick the STRING_TYPE reader instead of FULLTEXT, and WildcardWeight would enumerate terms from the wrong index directory, returning empty results. Fix: extend the query type upgrade in FieldReaderResolver::resolve() to also cover WILDCARD_QUERY, so wildcard patterns correctly use the FULLTEXT index on variant subcolumns.
hello-stephen
commented
Feb 21, 2026
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
airborne12
commented
Feb 21, 2026
run buildall |
doris-robot
commented
Feb 21, 2026
TPC-H: Total hot run time: 28664 ms |
doris-robot
commented
Feb 21, 2026
TPC-DS: Total hot run time: 184535 ms |
doris-robot
commented
Feb 21, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
…zzy test failure The CI fuzzy testing randomly enables default_variant_enable_doc_mode, which stores variant data in doc mode where inverted index is unavailable, causing all search() queries to return empty results. Explicitly set default_variant_enable_doc_mode = false to ensure the inverted index is available for the test.
airborne12
commented
Feb 21, 2026
run buildall |
doris-robot
commented
Feb 21, 2026
TPC-H: Total hot run time: 28636 ms |
doris-robot
commented
Feb 21, 2026
TPC-DS: Total hot run time: 183645 ms |
hello-stephen
commented
Feb 21, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 21, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 21, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 23, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
…compilation (apache#7378) ## Summary - Cherry-pick apache#60793: Fix wildcard search queries (`*`, `?` patterns) on variant subcolumns returning empty results when using Lucene-mode `search()` function - Fix pre-existing BE unit test compilation errors on hubspot branch (9 test files) ## Changes ### Core fix (cherry-pick from apache#60793) - `be/src/vec/functions/function_search.cpp`: Extend `FieldReaderResolver::resolve()` to upgrade `WILDCARD_QUERY` (in addition to `EQUAL_QUERY`) to `MATCH_ANY_QUERY` for variant subcolumns with analyzer-based indexes, so wildcard patterns correctly use the FULLTEXT index - New regression test `test_search_variant_wildcard.groovy` covering TERM baseline + wildcard patterns (`*ith`, `sm*th`, `sm?th`, `smith*`, etc.) ### BE UT compilation fixes (pre-existing issues) - `need_update_lru_blocks_test.cpp`: `KeyMeta::tablet_id` field removed - `occur_boolean_query_*_test.cpp`: Adapt to scoring-disabled hubspot branch (`weight()` no-arg, `DoNothingCombiner` returns 1.0F, `AllQuery` default constructor) - `segment_postings_test.cpp`: Add missing `override` keywords - `all_query_test.cpp`: Fix `AllQuery` constructor usage - `column_array_test.cpp`: `PrimitiveType` → `TypeIndex`, remove deleted APIs, fix duplicate test name link conflict - `vexpr_evalute_inverted_index_test.cpp`: Add missing `#include` - `function_string_test.cpp`: `PrimitiveType` → `TypeIndex` ## Test plan - [x] BE production build passes - [x] FE build passes - [x] BE UT compilation and linking passes - [x] 108 inverted index related unit tests pass - [x] 24/24 search regression tests pass (including new `test_search_variant_wildcard`) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…olumns with multi-index (#60834) ### What problem does this PR solve? Related PR: #60793 Problem Summary: Wildcard queries (*, ?) on variant subcolumns with multiple inverted indexes (one without analyzer, one with standard/lowercase analyzer) previously returned empty results even when regular TERM search worked correctly. This PR adds a regression test (`test_search_variant_wildcard_custom_analyzer`) to verify the fix from #60793 with a more realistic scenario matching the HubSpot contacts table pattern: - Variant column with dual field_pattern indexes (none + standard/lowercase) - firstname/lastname stored in variant subcolumns (`string_8`, `string_17`) - 13 test cases covering: TERM baseline, leading/middle/trailing/single-char wildcards, cross-field AND, and match-all wildcard
…olumns with multi-index (#60834) ### What problem does this PR solve? Related PR: #60793 Problem Summary: Wildcard queries (*, ?) on variant subcolumns with multiple inverted indexes (one without analyzer, one with standard/lowercase analyzer) previously returned empty results even when regular TERM search worked correctly. This PR adds a regression test (`test_search_variant_wildcard_custom_analyzer`) to verify the fix from #60793 with a more realistic scenario matching the HubSpot contacts table pattern: - Variant column with dual field_pattern indexes (none + standard/lowercase) - firstname/lastname stored in variant subcolumns (`string_8`, `string_17`) - 13 test cases covering: TERM baseline, leading/middle/trailing/single-char wildcards, cross-field AND, and match-all wildcard
…y results (apache#60793) ### What problem does this PR solve? Related PR: apache#60782 Problem Summary: When `search()` DSL uses wildcard patterns (e.g. `*ith`, `sm*th`, `sm?th`) on variant subcolumns with analyzer-based indexes (field_pattern), the queries return empty results even though regular TERM search works correctly. **Root cause:** In `FieldReaderResolver::resolve()`, only `EQUAL_QUERY` was upgraded to `MATCH_ANY_QUERY` for variant subcolumns with analyzer-based indexes. `WILDCARD_QUERY` was not upgraded, so `select_best_reader()` picked the `STRING_TYPE` reader instead of `FULLTEXT`. `WildcardWeight` then enumerated terms from the wrong (untokenized) index directory, finding no matches. **Fix:** Extend the query type upgrade condition to also cover `WILDCARD_QUERY`, so wildcard patterns correctly use the FULLTEXT index on variant subcolumns. Also fix a misleading comment in `inverted_index_iterator.cpp` where `is_equal_query()` was described as handling WILDCARD/REGEXP but actually only checks `EQUAL_QUERY`.
…olumns with multi-index (apache#60834) ### What problem does this PR solve? Related PR: apache#60793 Problem Summary: Wildcard queries (*, ?) on variant subcolumns with multiple inverted indexes (one without analyzer, one with standard/lowercase analyzer) previously returned empty results even when regular TERM search worked correctly. This PR adds a regression test (`test_search_variant_wildcard_custom_analyzer`) to verify the fix from apache#60793 with a more realistic scenario matching the HubSpot contacts table pattern: - Variant column with dual field_pattern indexes (none + standard/lowercase) - firstname/lastname stored in variant subcolumns (`string_8`, `string_17`) - 13 test cases covering: TERM baseline, leading/middle/trailing/single-char wildcards, cross-field AND, and match-all wildcard
…o branch-4.0 Squashed backport of the following master PRs: - apache#59747 [fix](search) Make AND/OR/NOT operators case-sensitive in search DSL - apache#60654 [refactor](search) Refactor SearchDslParser to single-phase ANTLR parsing and fix ES compatibility issues - apache#60782 [fix](search) Upgrade query type for variant subcolumns with analyzer-based indexes - apache#60784 [fix](search) Fix MATCH_ALL_DOCS query failing in multi-field search mode - apache#60786 [feat](search) Support field-grouped query syntax field:(term1 OR term2) - apache#60790 [fix](search) Add searcher cache reuse and DSL result cache for search() function - apache#60793 [fix](search) Fix wildcard query on variant subcolumns returning empty results - apache#60798 [fix](search) Use FE-provided analyzer key for multi-index columns in search() - apache#60814 [fix](search) Fix implicit conjunction incorrectly modifying preceding term in lucene mode - apache#60834 [test](search) Add regression test for wildcard query on variant subcolumns with multi-index - apache#60873 [fix](search) fix MATCH_ALL_DOCS losing occur attribute in multi-field expansion - apache#60891 [fix](search) inject MATCH_ALL_DOCS for multi-MUST_NOT queries in lucene mode
…o branch-4.0 Squashed backport of the following master PRs: - apache#59747 [fix](search) Make AND/OR/NOT operators case-sensitive in search DSL - apache#60654 [refactor](search) Refactor SearchDslParser to single-phase ANTLR parsing and fix ES compatibility issues - apache#60782 [fix](search) Upgrade query type for variant subcolumns with analyzer-based indexes - apache#60784 [fix](search) Fix MATCH_ALL_DOCS query failing in multi-field search mode - apache#60786 [feat](search) Support field-grouped query syntax field:(term1 OR term2) - apache#60790 [fix](search) Add searcher cache reuse and DSL result cache for search() function - apache#60793 [fix](search) Fix wildcard query on variant subcolumns returning empty results - apache#60798 [fix](search) Use FE-provided analyzer key for multi-index columns in search() - apache#60814 [fix](search) Fix implicit conjunction incorrectly modifying preceding term in lucene mode - apache#60834 [test](search) Add regression test for wildcard query on variant subcolumns with multi-index - apache#60873 [fix](search) fix MATCH_ALL_DOCS losing occur attribute in multi-field expansion - apache#60891 [fix](search) inject MATCH_ALL_DOCS for multi-MUST_NOT queries in lucene mode
… bug fixes (#61028) ### What problem does this PR solve? Squashed backport of all search() function improvements and bug fixes from master to branch-4.0. This PR combines the following master PRs into a single backport: | Master PR | Type | Description | |-----------|------|-------------| | #59747 | fix | Make AND/OR/NOT operators case-sensitive in search DSL | | #60654 | refactor | Refactor SearchDslParser to single-phase ANTLR parsing and fix ES compatibility issues | | #60782 | fix | Upgrade query type for variant subcolumns with analyzer-based indexes | | #60784 | fix | Fix MATCH_ALL_DOCS query failing in multi-field search mode | | #60786 | feat | Support field-grouped query syntax field:(term1 OR term2) | | #60790 | fix | Add searcher cache reuse and DSL result cache for search() function | | #60793 | fix | Fix wildcard query on variant subcolumns returning empty results | | #60798 | fix | Use FE-provided analyzer key for multi-index columns in search() | | #60814 | fix | Fix implicit conjunction incorrectly modifying preceding term in lucene mode | | #60834 | test | Add regression test for wildcard query on variant subcolumns with multi-index | | #60873 | fix | fix MATCH_ALL_DOCS losing occur attribute in multi-field expansion | | #60891 | fix | inject MATCH_ALL_DOCS for multi-MUST_NOT queries in lucene mode | ### Release note Backport search() function improvements including DSL parser refactoring, multi-field search fixes, variant subcolumn support, query caching, and field-grouped query syntax. ### Check List (For Author) - Test - [x] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [x] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [x] Yes. New search() function features and bug fixes backported from master. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
What problem does this PR solve?
Related PR: #60782
Problem Summary:
When
search()DSL uses wildcard patterns (e.g.*ith,sm*th,sm?th) on variant subcolumns with analyzer-based indexes (field_pattern), the queries return empty results even though regular TERM search works correctly.Root cause: In
FieldReaderResolver::resolve(), onlyEQUAL_QUERYwas upgraded toMATCH_ANY_QUERYfor variant subcolumns with analyzer-based indexes.WILDCARD_QUERYwas not upgraded, soselect_best_reader()picked theSTRING_TYPEreader instead ofFULLTEXT.WildcardWeightthen enumerated terms from the wrong (untokenized) index directory, finding no matches.Fix: Extend the query type upgrade condition to also cover
WILDCARD_QUERY, so wildcard patterns correctly use the FULLTEXT index on variant subcolumns. Also fix a misleading comment ininverted_index_iterator.cppwhereis_equal_query()was described as handling WILDCARD/REGEXP but actually only checksEQUAL_QUERY.Release note
Fix wildcard search queries (
*,?patterns) on variant subcolumns returning empty results when using Lucene-modesearch()function.Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)