Skip to content

[pick](expr) pick expr modify to branch4.0 - #59241

Merged
yiguolei merged 6 commits into
apache:branch-4.0from
Mryange:pick-expr-to-4.0
Dec 23, 2025
Merged

[pick](expr) pick expr modify to branch4.0#59241
yiguolei merged 6 commits into
apache:branch-4.0from
Mryange:pick-expr-to-4.0

Conversation

@Mryange

@MryangeMryange commented Dec 22, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

pick

#56905

#58125

#58692

#58578

#58710

#58841

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
[refine](condition) Change conditional functions to native expr (apac…
912e575
…he#58125)
Previously our conditional functions used the function framework.
For example: if(cond, exprA, exprB). Before executing if, cond, exprA,
and exprB were already evaluated.
By switching to expr, we can choose to evaluate either exprA or exprB
based on the actual value of cond.
Switching to expr also paves the way to support short-circuit evaluation
in the future.
@Mryange
[fix](core) Fix count_true_with_notnull failing to correctly handle C…
4fcf889
…onst (Nullable) cases. (apache#58692)
apache#58125
```
F20251203 19:47:55.046221 2067836 status.h:467] Bad cast from type:doris::vectorized::ColumnConst* to doris::vectorized::ColumnVector<(doris::PrimitiveType)2> const*
*** Check failure stack trace: ***
@ 0x55d7d5bc798f google::LogMessage::SendToLog()
@ 0x55d7d5bbdfa0 google::LogMessage::Flush()
@ 0x55d7d5bc1699 google::LogMessageFatal::~LogMessageFatal()
@ 0x55d7a87a211b doris::Status::FatalError<>()
@ 0x55d7c8732ed6 _ZZ11assert_castIPKN5doris10vectorized12ColumnVectorILNS0_13PrimitiveTypeE2EEEL18TypeCheckOnRelease1EPKNS1_7IColumnEET_OT1_ENKUlOSB_E_clISA_EES6_SE_
@ 0x55d7c87322e4 assert_cast<>()
@ 0x55d7c96b4334 doris::vectorized::VConditionExpr::count_true_with_notnull()
@ 0x55d7c96c2edb doris::vectorized::VectorizedIfExpr::execute_column()
@ 0x55d7b02f867d doris::vectorized::VExpr::execute()
@ 0x55d7c94e1a4b doris::vectorized::VExpr::get_const_col()
@ 0x55d7c94e0cfe doris::vectorized::VExpr::open()
@ 0x55d7c96b246e doris::vectorized::VConditionExpr::open()
@ 0x55d7c9881761 doris::vectorized::VExprContext::open()
@ 0x55d7c94eaeac doris::vectorized::VExpr::open()
```
@Mryange
[refine](expr) Expr execution does not depend on the block's rows. (a…
1b1b30c
…pache#58578)
Previously, when executing an expr we relied on block.rows() to
determine the number of rows to return. That required guaranteeing the
block was not empty and that the first column (since block.rows() is
implemented from it) had the final desired row count. Now we pass a
count parameter to indicate the size each expr should return for its
Column.
before
```C++
vectorized::Block tmp_block;
tmp_block.insert({vectorized::ColumnUInt8::create(1),
std::make_shared<vectorized::DataTypeUInt8>(), ""});
RETURN_IF_ERROR(_const_expr_lists[_const_expr_list_idx][i]->execute(&tmp_block,
&result_list[i]));
```
now
```C++
RETURN_IF_ERROR(_const_expr_lists[_const_expr_list_idx][i]->execute_const_expr(
tmp_block_columns[i]));
Status VExprContext::execute_const_expr(ColumnWithTypeAndName& result) {
Status st;
RETURN_IF_CATCH_EXCEPTION({ st = _root->execute_column(this, nullptr, 1, result.column); });
```
@Mryange
[chore](exec) BE constant folding no longer needs to construct a temp…
15e7272
…orary block (apache#58710)
Also, this PR fixes a bug: previously BE constant folding did not pass
whether casts should be performed in strict mode.
```
W20251205 13:32:09.197405 512888 internal_service.cpp:1608] exec fold constant expr failed, errmsg=[INTERNAL_ERROR]ColumnWithTypeAndName check column type failed, column name: (CAST String(String) TO IPv4), type: IPv4, column: Const(Nullable(IPV4)) , error: [INTERNAL_ERROR]Column type Const(Nullable(IPV4)) is not compatible with data type IPv4 0# doris::Status doris::vectorized::IDataType::check_column_non_nested_type<doris::vectorized::ColumnVector<(doris::PrimitiveType)36> >(doris::vectorized::IColumn const&) const at /root/doris/be/src/common/status.h:0
1# doris::vectorized::DataTypeNumberBase<(doris::PrimitiveType)36>::check_column(doris::vectorized::IColumn const&) const at /root/doris/be/src/vec/data_types/data_type_number_base.cpp:239
2# doris::vectorized::ColumnWithTypeAndName::check_type_and_column_match() const at /root/doris/be/src/vec/core/column_with_type_and_name.cpp:0
3# doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) at /root/doris/be/src/vec/exprs/vexpr_context.cpp:0
4# doris::FoldConstantExecutor::fold_constant_vexpr(doris::TFoldConstantParams const&, doris::PConstantExprResult*) at /root/doris/be/src/common/status.h:0
5# std::_Function_handler<void (), doris::PInternalService::fold_constant_expr(google::protobuf::RpcController*, doris::PConstantExprRequest const*, doris::PConstantExprResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /root/doris/be/src/common/status.h:0
```
@Mryange
[chore](code)Remove the unused member variable _getting_const_col and…
1824bea
… get_const_col no longer constructs a block. (apache#58841)
Currently this variable is only used in DCHECK. Whether to apply the
const optimization (is_const_and_have_executed) does not depend on this
variable.
```
bool is_const_and_have_executed() const {
return (is_constant() && (_constant_col != nullptr));
}
```
We can determine whether we're currently executing a constant expression
by checking if block is nullptr.
get_const_col no longer constructs a block
```
int result = -1;
Block block;
// If block is empty, some functions will produce no result. So we insert a column with
// single value here.
block.insert({ColumnUInt8::create(1), std::make_shared<DataTypeUInt8>(), ""});
RETURN_IF_ERROR(execute(context, &block, &result));
```
now
```
ColumnPtr result;
RETURN_IF_ERROR(execute_column(context, nullptr, 1, result));
_constant_col = std::make_shared<ColumnPtrWrapper>(result);
```
@Thearas

Thearas commented Dec 22, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@MryangeMryange changed the title [pick](expr) pick expr modify[pick](expr) pick expr modify to branch4.0Dec 22, 2025
@Mryange

Mryange commented Dec 22, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Dec 22, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 7.79% (58/745) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage53.07% (18474/34810)
Line Coverage38.75% (170214/439316)
Region Coverage33.57% (131680/392260)
Branch Coverage34.43% (56807/164995)

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2025
@github-actions

github-actionsBot commented Dec 23, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Dec 23, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yiguolei
yiguolei merged commit 82f1099 into apache:branch-4.0Dec 23, 2025
23 of 27 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Mryange@Thearas@doris-robot@yiguolei