Uh oh!
There was an error while loading. Please reload this page.
[fix](point query) fix point query IllegalStateException on LIST partition - #66416
[fix](point query) fix point query IllegalStateException on LIST partition#66416hulincup wants to merge 9 commits into
Conversation
Point query (short-circuit) on a table with PARTITION BY LIST and multiple tablets per partition threw IllegalStateException because the tablet list was not pruned to a single tablet. In OlapScanNode.computeTabletInfo, point queries forced the legacy HashDistributionPruner path (isNereids && !isPointQuery), which relies on columnFilters populated from legacy conjuncts. For a nereids-planned direct point query those column filters are not populated, so the pruner returned all tablets of the matched partition and PointQueryExecutor.setScanRangeLocations hit its single-tablet checkState. Use the nereids-pruned tablet set for point queries when it has already been pruned to a single tablet (direct query, literals known at planning time). When the pruned set still contains multiple tablets (prepared statement, parameter values unknown at planning time), keep the legacy runtime-prune path so that path is not regressed. Fixesapache#66030
hello-stephen
commented
Aug 4, 2026
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
…ssion test Address review feedback on apache#66416: - Hoist useNereidsPrune out of the per-partition loop (loop-invariant; isNereids/isPointQuery are already hoisted above it). - Add a prepared-statement point-query case to the regression test to guard the no-regression property: a prepared point query (parameter value unknown at planning) keeps taking the legacy runtime-prune path and must still resolve to the correct tablet.
hulincup
commented
Aug 4, 2026
Thanks for the checklist. Describing this PR per the points (the PR description above covers these in detail; summarizing here): 1. Problem fixed & how 2. Behavior modified (previous vs. now, impact)
3. Features added — N/A. I could not build/run FE locally; verification relies on CI. The new regression test ( |
eldenmoon
commented
Aug 5, 2026
run buildall |
eldenmoon
commented
Aug 5, 2026
/review |
Codex automated review failed and did not complete. Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 8th, 2026 3:37 AM. The selected account is excluded until 2026-08-08T03:37:00Z. Please trigger /review again; another configured account may be available. |
There was a problem hiding this comment.
Pull request overview
Fixes a Nereids-planned short-circuit point query crash on PARTITION BY LIST tables with multiple tablets per partition by allowing Nereids’ pre-pruned tablet set to be used for point queries when it is already reduced to a single tablet. Adds a regression test covering both the direct point-query path (fixed) and the server-side prepared-statement path (must not regress).
Changes:
- Adjust tablet distribution pruning selection in
OlapScanNode.computeTabletInfo()to use Nereids pruning for point queries only whennereidsPrunedTabletIds.size() == 1. - Add regression coverage for LIST partition + multiple buckets + short-circuit point query, plus a prepared-statement variant to guard legacy runtime pruning.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java | Switches point-query distribution pruning to use Nereids-pruned tablet IDs when already singular to avoid multi-tablet scan ranges causing IllegalStateException. |
regression-test/suites/point_query_p0/test_point_query_list_partition.groovy | New regression suite reproducing the LIST-partition point-query failure and validating both direct and prepared-statement execution paths. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def explain = sql """EXPLAIN SELECT * FROM ${tableName} WHERE pk = 'abcd' AND _id = 1""" | ||
| assertTrue(explain.toString().contains("SHORT-CIRCUIT")) |
There was a problem hiding this comment.
Good point — done in 545f4ec. Switched to the regression framework's explain { sql(...); contains "SHORT-CIRCUIT" } helper, matching the idiom used across point_query_p0 (test_point_query.groovy, load_ck.groovy).
| assertEquals(1, rowCount) | ||
| } | ||
| sql """DROP TABLE IF EXISTS ${tableName}""" |
There was a problem hiding this comment.
I'll keep the trailing DROP TABLE IF EXISTS. The other point_query_p0 suites that partition (test_point_query_partition.groovy:117, test_point_query_timezone.groovy:70) drop at the end too, so this matches the existing convention rather than the keep-table-around one.
Address Copilot review feedback on apache#66416: replace the brittle sql("EXPLAIN ...") + toString().contains(...) with the regression framework's explain { sql(...); contains ... } helper, matching the idiom used across the point_query_p0 suite (e.g. test_point_query.groovy, load_ck.groovy).
hulincup
commented
Aug 5, 2026
Thanks @eldenmoon. On On Copilot's two line comments are addressed below. |
eldenmoon
commented
Aug 5, 2026
run buildall |
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
hello-stephen
commented
Aug 5, 2026
FE UT Coverage ReportIncrement line coverage |
hulincup
commented
Aug 5, 2026
Update on buildall: thanks @eldenmoon — the I can't reach the TeamCity log directly (auth required). The FE change in this PR is a one-line logic tweak: hoisting |
hulincup
commented
Aug 5, 2026
run buildall |
hulincup
commented
Aug 5, 2026
COMPILE (DORIS_COMPILE) failed again on the rerun (buildId 1015987, 05:59), so it's not a flake. But I believe it's a BE/master compile issue, not this PR:
I can't reach the TeamCity log (auth required). Could you check the COMPILE build log for buildId 1015987 and confirm whether it's a BE/master failure? If it is on the FE side I'll fix immediately, but the change is type-safe and FE UT passing strongly suggests FE compiles fine. |
hulincup
commented
Aug 5, 2026
Cross-checking today COMPILE results across PRs — 4 of 5 failed in the same window, including ones that only touch FE or a single BE test file:
This points to a systemic master/CI COMPILE issue today, not this PR (which has zero BE/C++ changes and FE UT passed on the prior run). FE UT is rerunning now and will confirm FE compiles. Could a committer check whether the COMPILE environment needs a reset? Happy to fix if the FE UT rerun surfaces anything. |
hello-stephen
commented
Aug 5, 2026
FE UT Coverage ReportIncrement line coverage |
eldenmoon
commented
Aug 5, 2026
run buildall |
hello-stephen
commented
Aug 5, 2026
FE UT Coverage ReportIncrement line coverage |
hello-stephen
commented
Aug 5, 2026
FE Regression Coverage ReportIncrement line coverage |
hulincup
commented
Aug 6, 2026
The rerun (thanks @eldenmoon) turned most checks green: COMPILE, FE UT, vault_p0, External Regression, and the coverage reports all pass. Four regression jobs still fail, and I want to narrow down whether they are mine:
I cannot reach the TeamCity logs (auth required) to get the failing test names. The FE change only tightens tablet-prune path selection for nereids point queries already pruned to a single tablet, and FE UT passes, but the "new failed" tag makes me want to confirm. Could you share the failing test names so I can tell whether my new regression test or the prune change is involved, versus a master flake? Happy to fix immediately once I know which test. |
eldenmoon
commented
Aug 6, 2026
you can "Log in as guest"
![]() |
In addKeyTuples, columnExpr.get(column.getName()) could return null when a key column has no matching literal conjunct on the scan node (e.g. a nereids slot columnName that does not match the base schema column name). The null literalExpr then fell through to ExprToThriftVisitor.treeToThrift and threw NullPointerException on expr.getType(), masking the root cause. Throw a diagnostic TException naming the unmatched key column and the available conjunct columns instead, so the next run reveals the exact column-name mismatch.
hulincup
commented
Aug 7, 2026
Thanks — logged in as guest and pulled the failing tests. Triaging: My new Root-cause trace (static): the NPE is thrown at My I've pushed a defensive check (commit 7e5ec29) that throws a diagnostic TException — naming the unmatched key column and the available conjunct columns — instead of the NPE, so the next run reveals the exact column-name mismatch. I still need to confirm why the conjunct slot columnName doesn't match the base schema column name on this LIST-partitioned table, then fix that. Existing The rest are unrelated — search_score_cache, compaction, eager_agg, eliminate_inner, temp_table, multi_leading, mow_compact, performance (deploy step). Pre-existing in their own areas. |
eldenmoon
commented
Aug 7, 2026
/review |
Codex automated review failed and did not complete. Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-08T03:32:00Z. Please trigger /review again after that time. |
hulincup
commented
Aug 7, 2026
run buildall |
hello-stephen
commented
Aug 7, 2026
TPC-H: Total hot run time: 28664 ms |
hello-stephen
commented
Aug 7, 2026
TPC-DS: Total hot run time: 158600 ms |
hello-stephen
commented
Aug 7, 2026
ClickBench: Total hot run time: 23.87 s |
hello-stephen
commented
Aug 7, 2026
FE Regression Coverage ReportIncrement line coverage |
hulincup
commented
Aug 19, 2026
run buildall |
Hi @hulincup, thank you for working on this! |


Motivation
Fixes#66030.
A short-circuit point query on a table with
PARTITION BY LISTand more than one tablet per partition threwIllegalStateException(msg: null):Repro (from the issue):
Root cause
In
OlapScanNode.computeTabletInfo, the distribution prune flag isisNereids && !isPointQuery, which forces point queries onto the legacyHashDistributionPrunerpath. That path prunes usingcolumnFilters, which is populated bycomputeColumnsFilterfrom the legacyconjuncts. For a nereids-planned direct point query the conjuncts are not in the legacy form that feedscolumnFilters, so the pruner returns all tablets of the matched partition.PointQueryExecutor.setScanRangeLocationsthen hitscheckState(scanTabletIds.size() == 1)with an un-pruned list and crashes. The bug is masked when a partition has a single tablet (e.g.BUCKETS 1), so it only surfaces withBUCKETS AUTO/ multiple buckets — which is why LIST-partition users hit it.Change
Use the nereids-pruned tablet set for point queries when it has already been pruned to a single tablet (the direct-query case, where the key literals are known at planning time and
PhysicalPlanTranslatorpopulatesnereidsPrunedTabletIdsfrom the nereids plan'sselectedTabletIds). When the pruned set still contains multiple tablets (a prepared-statement point query whose parameter values are unknown at planning time), keep the legacy runtime-prune path, so that path is not regressed.Behavior matrix:
!isPointQueryis true).nereidsPrunedTabletIdshas 1 tablet → uses nereids path → pruned to 1 → fixed.nereidsPrunedTabletIdshas multiple tablets (value unknown at planning) → falls back to legacy runtime prune → unchanged.Files
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java— the prune-flag change.regression-test/suites/point_query_p0/test_point_query_list_partition.groovy— new regression test reproducing the issue (LIST partition + BUCKETS 3 + short-circuit point query).Verification
I could not build/run FE locally for this change; verification relies on CI. The new regression test reproduces the original failure (it asserts the query returns 1 row instead of throwing) and the existing
point_query_p0suites guard the non-regression side.Test plan
point_query_p0/test_point_query_list_partitionpasses.point_query_p0/*(incl.test_point_query_partition,test_point_query, prepared-statement paths) pass unchanged.