[Feature](execution) support adjuct filter order by cost by BiteTheDDDDt · Pull Request #60079 · apache/doris

BiteTheDDDDt · 2026-01-20T14:05:52Z

What problem does this PR solve?

tpch external 187s -> 180s
tpcds external 380s -> 370s

This pull request introduces a mechanism to estimate and utilize the execution cost of conjunct (predicate) expressions, enabling the system to dynamically reorder conjuncts for improved query performance when the new session variable enable_adjust_conjunct_order_by_cost is enabled. The changes span the addition of cost estimation methods to expression classes, integration of cost-based sorting in several operator initialization and runtime paths, and the introduction of the new session variable in both backend and frontend.

Key changes include:

Predicate Execution Cost Estimation

Added a virtual execute_cost() method to the base expression class VExpr and implemented or overridden it in various expression types (VLiteral, VSlotRef, VColumnRef, VCompoundPred, VectorizedFnCall, VRuntimeFilterWrapper, and function classes like FunctionComparison), providing a mechanism to estimate the computational cost of evaluating each predicate. ([[1]](https://github.com/apache/doris/pull/60079/files#diff-4150da5bf9ad89014e38f3a83e742a52468701b366ca4e4c6b09068c7dac283aR279-R286), [[2]](https://github.com/apache/doris/pull/60079/files#diff-17d34fd4d8b4a91c6eb1e3238715ce5e1c58b4968faea7d88ed22cfe0e348a91R59-R60), [[3]](https://github.com/apache/doris/pull/60079/files#diff-7eee9b0f155e5dd8d9d3a10dd72f7fb395cc0f65ff7f5de9ff5166599281d33fR75-R76), [[4]](https://github.com/apache/doris/pull/60079/files#diff-32461b08211d014e6636fd1cba54dc37cfb4916c13b47ac8ad36cb989a0b2031R92-R93), Fd3c42dfL347R347, [[5]](https://github.com/apache/doris/pull/60079/files#diff-01445f9862bc32606a103e5801fca9bd284f7dc2b9e1a6b7f28ec385d2ef2f24R67), [[6]](https://github.com/apache/doris/pull/60079/files#diff-dca2c0f573793d20c2dba5fe846bbadd82080fdea00051785cf0ea1ca41578bfR643-R655), [[7]](https://github.com/apache/doris/pull/60079/files#diff-93a2e196a10e91c528edefa02f02abb5b6ee621a3e1cbcc2d6a5ca7075a6061dR173-R174), [[8]](https://github.com/apache/doris/pull/60079/files#diff-f698d8639284042c1bb81cee0ae26a702114342abd4fa3bdfee4a51f62744f28R275-R276), [[9]](https://github.com/apache/doris/pull/60079/files#diff-f7878a776d3726dd835f26223f373f4d2e54f5080fa239868438adbe8bb911ffR67-R68), [[10]](https://github.com/apache/doris/pull/60079/files#diff-08b2eb65f6fe3da674c54840194c031f1309e06587938f5d31b04f2f9f4e9c68L204-R206), [[11]](https://github.com/apache/doris/pull/60079/files#diff-c0aab1f4c7a9ee63428155fea5d63afda8b47794f881392f71a47a619b334761R476-R484))

Cost-Based Predicate Reordering

Integrated cost-based sorting of conjuncts (using std::ranges::sort and execute_cost()) in several operator and scan initialization and runtime methods, so that when enable_adjust_conjunct_order_by_cost is true, predicates are evaluated in increasing order of estimated cost. This affects scan operators, aggregation operators, and Parquet group readers. ([[1]](https://github.com/apache/doris/pull/60079/files#diff-8922a03a948ba6a44dda51be21f535e0f5b8725b37b2f10facc90f2476b82aebL202-L214), [[2]](https://github.com/apache/doris/pull/60079/files#diff-ac38473f151ccd99db4ed80dfbfa176f4f957ae2f5335ce8fbb4f5dbb0e932e9L80-R87), [[3]](https://github.com/apache/doris/pull/60079/files#diff-ac38473f151ccd99db4ed80dfbfa176f4f957ae2f5335ce8fbb4f5dbb0e932e9R194-R199), [[4]](https://github.com/apache/doris/pull/60079/files#diff-d09145594c823444cca71879eb6515950211b548d7cbef65f0caf5c1d88f296fR191-R196))

Session Variable and Query Option

Introduced the session variable enable_adjust_conjunct_order_by_cost (default true) to control whether cost-based predicate reordering is enabled, and ensured it is forwarded and included in query options for both backend and frontend. ([[1]](https://github.com/apache/doris/pull/60079/files#diff-efeb89760ec70cdad5ae9c6169e213b9efa84fccb6ede73f626bf6583bd82333R128-R132), [[2]](https://github.com/apache/doris/pull/60079/files#diff-b8d4d6b7e84c9f48322536f241d11f98401524e08aba36b0c92be9589f43a4a3R3090-R3092), [[3]](https://github.com/apache/doris/pull/60079/files#diff-b8d4d6b7e84c9f48322536f241d11f98401524e08aba36b0c92be9589f43a4a3R5065))

Code Cleanup and Refactoring

Removed legacy handling of vconjunct in aggregation and streaming aggregation operators, simplifying conjunct management and aligning with the new cost-based mechanism. ([[1]](https://github.com/apache/doris/pull/60079/files#diff-df21df88282293d1bbe3223e0d95dcc5afc01da571e3a7c1d9dc9377a758d5daL737-R737), [[2]](https://github.com/apache/doris/pull/60079/files#diff-5f2882c1f711fc0954459c6b98a1dcde9b688bb5634be71ce6f585332d8b6497L828), [[3]](https://github.com/apache/doris/pull/60079/files#diff-81893b8013f0ade85a4aca0bcf807655fc08a57e7a6cc8531278cdc5330fae9eL265))
Minor improvements such as renaming profile info strings for clarity (e.g., "RemainedDownPredicates" to "RemainedPredicates"). ([be/src/pipeline/exec/scan_operator.cppL315-R327](https://github.com/apache/doris/pull/60079/files#diff-ac38473f151ccd99db4ed80dfbfa176f4f957ae2f5335ce8fbb4f5dbb0e932e9L315-R327))

These changes collectively enable more efficient predicate evaluation, which can reduce query execution time, especially for complex queries with multiple predicates of varying computational cost.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-01-20T14:06:00Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

BiteTheDDDDt · 2026-01-20T14:07:45Z

run buildall

BiteTheDDDDt · 2026-01-20T14:09:17Z

run buildall

Copilot

Pull request overview

This PR introduces a cost-based mechanism to reorder Parquet filter conjuncts, controlled by a new session/thrift query option, and adds plumbing to estimate expression/function execution cost in the vectorized engine.

Changes:

Added enable_adjust_conjunct_order_by_cost to TQueryOptions and exposed it as a session variable forwarded to BE.
Introduced execute_cost() hooks across vectorized expressions/contexts and function interfaces.
Sorted Parquet row group _filter_conjuncts by estimated execution cost when the option is enabled.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
gensrc/thrift/PaloInternalService.thrift	Adds new thrift query option field to control cost-based conjunct ordering.
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java	Adds session variable and forwards it into `TQueryOptions` during serialization.
be/src/vec/functions/function.h	Introduces function-level `execute_cost()` API.
be/src/vec/functions/functions_comparison.h	Assigns a low cost to comparison functions via `execute_cost()`.
be/src/vec/exprs/vexpr.h	Adds expression-level recursive `execute_cost()` default implementation.
be/src/vec/exprs/vexpr_context.h	Exposes `execute_cost()` on `VExprContext` and adjusts `root()` accessor.
be/src/vec/exprs/vexpr_context.cpp	Implements `VExprContext::execute_cost()`.
be/src/vec/exprs/vectorized_fn_call.h	Declares cost override for vectorized function call expressions.
be/src/vec/exprs/vectorized_fn_call.cpp	Implements function-call cost estimation.
be/src/vec/exprs/vruntimefilter_wrapper.h	Forwards `execute_cost()` to wrapped runtime filter expression.
be/src/vec/exec/format/parquet/vparquet_group_reader.cpp	Sorts Parquet filter conjuncts by cost when enabled.

Comments suppressed due to low confidence (1)

fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:5073

In toThrift(), this field is set twice: first via the direct setter, then again via reflection. If the goal is compatibility with older generated thrift classes, the direct call defeats that (it won’t compile when the setter is missing). Please either keep only the direct setter (and drop reflection) or keep only the reflective set (and remove the direct call) to match the intended compatibility behavior.

 tResult.setEnableAdjustConjunctOrderByCost(enableAdjustConjunctOrderByCost);
// Set Iceberg write target file size
tResult.setIcebergWriteTargetFileSizeBytes(icebergWriteTargetFileSizeBytes);
return tResult;
}
public JSONObject toJson() throws IOException {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-20T14:16:01Z

+ if (_state->query_options().enable_adjust_conjunct_order_by_cost) {
+ std::ranges::sort(_filter_conjuncts,


Minor style: please keep spacing consistent with the rest of the file (if ().

Copilot · 2026-01-20T14:16:01Z

 public boolean enablePhraseQuerySequentialOpt = true;

+ @VariableMgr.VarAttr(name = "enable_adjust_conjunct_order_by_cost", needForward = true)
+ public boolean enableAdjustConjunctOrderByCost = true;


This session variable defaults to true, meaning predicate reordering by cost will be enabled by default. Since this can change evaluation order (and potentially error/short-circuit behavior), consider defaulting it to false (similar to other cost-based reordering toggles like enableJoinReorderBasedCost).
Suggested change
publicbooleanenableAdjustConjunctOrderByCost = true;
publicbooleanenableAdjustConjunctOrderByCost = false;

Copilot · 2026-01-20T14:16:02Z

+ virtual double execute_cost() const { return 1.0; }
+


Adding execute_cost() to IFunctionBase is fine, but note that most functions appear to be wrapped by DefaultFunction (be/src/vec/functions/function.h:436+). Since DefaultFunction doesn’t override/forward execute_cost(), any per-function overrides (e.g. in IFunction implementations) won’t be observed via FunctionBasePtr. Consider forwarding execute_cost() in DefaultFunction (or otherwise ensuring the wrapper preserves the underlying function’s cost).

@copilot open a new pull request to apply changes based on this feedback

hello-stephen · 2026-01-20T15:29:45Z

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-20T19:04:24Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

BiteTheDDDDt · 2026-01-21T08:09:27Z

run buildall

hello-stephen · 2026-01-21T09:18:54Z

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

BiteTheDDDDt · 2026-01-21T12:15:43Z

run buildall

hello-stephen · 2026-01-21T13:01:32Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.31% (1786/2252)
Line Coverage	64.74% (31802/49120)
Region Coverage	65.46% (15845/24207)
Branch Coverage	56.06% (8424/15028)

hello-stephen · 2026-01-21T13:37:36Z

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-21T16:01:07Z

BE UT Coverage Report

Increment line coverage 13.70% (10/73) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.43% (19112/36454)
Line Coverage	35.80% (177399/495542)
Region Coverage	32.29% (137146/424785)
Branch Coverage	33.20% (59330/178711)

hello-stephen · 2026-01-21T18:07:47Z

BE Regression && UT Coverage Report

Increment line coverage 91.78% (67/73) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.48% (25542/35733)
Line Coverage	54.05% (267539/494990)
Region Coverage	51.59% (221472/429292)
Branch Coverage	53.03% (95177/179476)

hello-stephen · 2026-01-21T18:19:16Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-22T03:46:52Z

BE Regression && UT Coverage Report

Increment line coverage 91.78% (67/73) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.49% (25545/35733)
Line Coverage	54.06% (267592/494990)
Region Coverage	51.63% (221656/429292)
Branch Coverage	53.04% (95199/179476)

hello-stephen · 2026-01-22T04:01:39Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

BiteTheDDDDt · 2026-01-27T11:54:10Z

run buildall

hello-stephen · 2026-01-27T13:02:12Z

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-27T15:44:26Z

BE UT Coverage Report

Increment line coverage 15.15% (10/66) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.71% (19253/36527)
Line Coverage	36.11% (179374/496701)
Region Coverage	32.58% (138841/426147)
Branch Coverage	33.51% (60072/179281)

hello-stephen · 2026-01-27T16:29:09Z

BE Regression && UT Coverage Report

Increment line coverage 87.69% (57/65) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.52% (25606/35804)
Line Coverage	54.16% (268699/496163)
Region Coverage	51.89% (223459/430655)
Branch Coverage	53.19% (95764/180051)

hello-stephen · 2026-01-27T16:41:37Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-28T03:50:16Z

BE Regression && UT Coverage Report

Increment line coverage 87.69% (57/65) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.52% (25606/35804)
Line Coverage	54.15% (268692/496163)
Region Coverage	51.88% (223412/430655)
Branch Coverage	53.18% (95752/180051)

hello-stephen · 2026-01-28T04:01:15Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

HappenLee

LGTM

github-actions · 2026-01-30T04:31:13Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-01-30T04:31:16Z

PR approved by anyone and no changes requested.

thrift: add enable_adjust_conjunct_order_by_cost to TQueryOptions fe: add session variable enable_adjust_conjunct_order_by_cost fe: forward enable_adjust_conjunct_order_by_cost via toThrift if supported update update format update Update be/src/vec/exprs/vexpr_context.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Update be/src/vec/exprs/vcompound_pred.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Update be/src/vec/exprs/vectorized_fn_call.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> update update

BiteTheDDDDt · 2026-02-11T06:29:50Z

run buildall

github-actions · 2026-02-11T09:07:34Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2026-02-11T13:17:45Z

BE Regression && UT Coverage Report

Increment line coverage 87.88% (58/66) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.76% (25959/36176)
Line Coverage	54.38% (271564/499375)
Region Coverage	51.64% (224956/435642)
Branch Coverage	53.24% (96890/181995)

hello-stephen · 2026-02-11T13:34:15Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-02-11T13:35:08Z

BE Regression && UT Coverage Report

Increment line coverage 87.88% (58/66) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.76% (25959/36176)
Line Coverage	54.38% (271564/499375)
Region Coverage	51.64% (224956/435642)
Branch Coverage	53.24% (96890/181995)

hello-stephen · 2026-02-11T13:43:04Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

tpch external 187s -> 180s tpcds external 380s -> 370s This pull request introduces a mechanism to estimate and utilize the execution cost of conjunct (predicate) expressions, enabling the system to dynamically reorder conjuncts for improved query performance when the new session variable `enable_adjust_conjunct_order_by_cost` is enabled. The changes span the addition of cost estimation methods to expression classes, integration of cost-based sorting in several operator initialization and runtime paths, and the introduction of the new session variable in both backend and frontend.

#58666 #61503 #60079 #60228 #60735

BiteTheDDDDt requested a review from zclllyybb as a code owner January 20, 2026 14:05

CopilotAI review requested due to automatic review settings January 20, 2026 14:05

Copilotstarted reviewing on behalf of BiteTheDDDDt January 20, 2026 14:06 View session

BiteTheDDDDt force-pushed the dev_0120_3 branch from 3f98a39 to ea53fcdCompare January 20, 2026 14:07

CopilotAI reviewed Jan 20, 2026
View reviewed changes

BiteTheDDDDt force-pushed the dev_0120_3 branch from 6c0f3b0 to 98e4685Compare January 21, 2026 07:37

BiteTheDDDDt requested a review from Copilot January 21, 2026 07:42

Copilotstarted reviewing on behalf of BiteTheDDDDt January 21, 2026 07:42 View session

CopilotAI reviewed Jan 21, 2026
View reviewed changes

Comment threadbe/src/vec/exprs/vexpr_context.cpp
Comment threadbe/src/vec/exprs/vcompound_pred.h Outdated
Comment threadbe/src/pipeline/exec/operator.cpp

BiteTheDDDDt force-pushed the dev_0120_3 branch from 357ca67 to 412d3efCompare January 21, 2026 08:09

BiteTheDDDDt changed the title ~~Dev 0120 3~~[Feature](execution) support adjuct filter order by costJan 22, 2026

BiteTheDDDDt force-pushed the dev_0120_3 branch from 412d3ef to 45ee513Compare January 27, 2026 11:53

HappenLee previously approved these changes Jan 30, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jan 30, 2026

github-actionsBot added the reviewed label Jan 30, 2026

BiteTheDDDDt dismissed HappenLee’s stale review via 78f9eedFebruary 11, 2026 06:24

BiteTheDDDDt force-pushed the dev_0120_3 branch from 45ee513 to 78f9eedCompare February 11, 2026 06:24

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Feb 11, 2026

zclllyybb approved these changes Feb 11, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 11, 2026

Mryange approved these changes Feb 12, 2026
View reviewed changes

BiteTheDDDDt merged commit e66fe27 into apache:masterFeb 12, 2026
31 of 33 checks passed

BiteTheDDDDt mentioned this pull request Mar 19, 2026
[Chore](pick) pick #58666 #60526 #60079 #60228 #60735 #61503
Merged

yiguolei added the dev/4.1.0-merged label Mar 20, 2026

yiguolei pushed a commit that referenced this pull request Mar 20, 2026
[Chore](pick) pick#58666#61503#60079#60228#60735(#61503)
b8a804d
#58666 #61503 #60079 #60228 #60735

		if (_state->query_options().enable_adjust_conjunct_order_by_cost) {
		std::ranges::sort(_filter_conjuncts,

	publicbooleanenableAdjustConjunctOrderByCost = true;
	publicbooleanenableAdjustConjunctOrderByCost = false;

Uh oh!

Conversation

BiteTheDDDDt commented Jan 20, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Predicate Execution Cost Estimation

Cost-Based Predicate Reordering

Session Variable and Query Option

Code Cleanup and Refactoring

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 20, 2026

Uh oh!

BiteTheDDDDt commented Jan 20, 2026

Uh oh!

BiteTheDDDDt commented Jan 20, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

CopilotAIJan 20, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIJan 20, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIJan 20, 2026

Choose a reason for hiding this comment

Uh oh!

BiteTheDDDDtJan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hello-stephen commented Jan 20, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Jan 20, 2026

FE Regression Coverage Report

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BiteTheDDDDt commented Jan 21, 2026

Uh oh!

hello-stephen commented Jan 21, 2026

FE UT Coverage Report

Uh oh!

BiteTheDDDDt commented Jan 21, 2026

Uh oh!

hello-stephen commented Jan 21, 2026

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Jan 21, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Jan 21, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jan 21, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jan 21, 2026

FE Regression Coverage Report

Uh oh!

hello-stephen commented Jan 22, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jan 22, 2026

FE Regression Coverage Report

Uh oh!

BiteTheDDDDt commented Jan 27, 2026

Uh oh!

hello-stephen commented Jan 27, 2026

BiteTheDDDDt commented Jan 20, 2026•
edited
Loading