Skip to content

[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (Part II) - #43218

Merged
zhangstar333 merged 1 commit into
apache:masterfrom
zclllyybb:cc2
Nov 11, 2024
Merged

[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (Part II)#43218
zhangstar333 merged 1 commit into
apache:masterfrom
zclllyybb:cc2

Conversation

@zclllyybb

@zclllyybbzclllyybb commented Nov 4, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

part 2 of #42930

two modifications:

  1. when parsing data of json array, cast to target type when parse int data no matter what source type is.
  2. in explode_numbers, the argument column type is certain Int32. so reduce some virtual function call.

@doris-robot

doris-robot commented Nov 4, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybbzclllyybb mentioned this pull request Nov 4, 2024
Merged
github-actions[bot]
github-actionsBot reviewed Nov 4, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/util/simd/bits.h
Comment threadbe/src/vec/exprs/table_function/table_function.h
Comment threadbe/src/vec/exprs/table_function/vexplode_json_array.h
Comment threadbe/src/vec/exprs/vcompound_pred.h
Comment threadbe/src/vec/exprs/vectorized_agg_fn.h
Comment threadbe/src/vec/exprs/vectorized_fn_call.h
Comment threadbe/src/vec/functions/array/function_array_index.h
Comment threadbe/src/vec/functions/function.h
Comment threadbe/src/vec/functions/function_date_or_datetime_computation.h
Comment threadbe/src/vec/functions/function_date_or_datetime_computation.h
BiteTheDDDDt pushed a commit that referenced this pull request Nov 8, 2024
@zclllyybb
[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (…
c86498d
…Part I) (#42930)
## Proposed changes
In this pr we refactored some base interface to avoid narrow cast with
data consistent kept.
We uniformed these type of numbers:
```
columns in Block - U32 (just use no more than U16)
Rows in Block - U64 (others U32, ColumnString64 U64)
column_id in expr arg - I32 (because -1 is meaningful)
child of Expr - U16 (FE use int32, have a restriction of 10000. adjustable but no guarantee for result)
```
So, for functions' argument column id, we use `uint32` to replace
`size_t`.
For rows, we use `size_t` to replace some `int`.
For child of expr, we cast it as `uint_16` to use, rather than `size_t`
as `vector::size()`
Refactor of table functions: #43218
@zclllyybb
zclllyybb marked this pull request as ready for review November 8, 2024 06:27
@zclllyybb

zclllyybb commented Nov 8, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 8, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@doris-robot

doris-robot commented Nov 8, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.90% (9864/26027)
Line Coverage: 29.10% (82273/282747)
Region Coverage: 28.24% (42336/149915)
Branch Coverage: 24.79% (21438/86474)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0c9e40fe19b6c36ca616964387fa8e479b9f0387_0c9e40fe19b6c36ca616964387fa8e479b9f0387/report/index.html

HappenLee
HappenLee approved these changes Nov 8, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 8, 2024
@github-actions

github-actionsBot commented Nov 8, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@zhangstar333
zhangstar333 merged commit 279c162 into apache:masterNov 11, 2024
@924060929924060929 mentioned this pull request Nov 15, 2024
Merged
16 tasks
924060929 added a commit that referenced this pull request Nov 18, 2024
@924060929
[fix](compile) fix macos compile failed (#44061)
ded2190
fix macos compile failed, introduced by #40813, #42930, #43218, #43289
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Nov 18, 2024
@924060929
[fix](compile) fix macos compile failed (apache#44061)
46390d5
fix macos compile failed, introduced by apache#40813, apache#42930, apache#43218, apache#43289
(cherry picked from commit ded2190)
924060929 added a commit that referenced this pull request Nov 18, 2024
@924060929
[fix](compile) fix macos compile failed (#44061) (#44201)
7d50da9
fix macos compile failed, introduced by #40813, #42930, #43218, #43289
@zclllyybb
zclllyybb deleted the cc2 branch January 21, 2025 02:33
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@zclllyybb@HappenLee
[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (…
20e1cab
…Part I) (apache#42930)
## Proposed changes
In this pr we refactored some base interface to avoid narrow cast with
data consistent kept.
We uniformed these type of numbers:
```
columns in Block - U32 (just use no more than U16)
Rows in Block - U64 (others U32, ColumnString64 U64)
column_id in expr arg - I32 (because -1 is meaningful)
child of Expr - U16 (FE use int32, have a restriction of 10000. adjustable but no guarantee for result)
```
So, for functions' argument column id, we use `uint32` to replace
`size_t`.
For rows, we use `size_t` to replace some `int`.
For child of expr, we cast it as `uint_16` to use, rather than `size_t`
as `vector::size()`
Refactor of table functions: apache#43218
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@zclllyybb@HappenLee
[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (…
7cbed00
…Part II) (apache#43218)
part 2 of apache#42930
two modifications:
1. when parsing data of json array, cast to target type when parse int
data no matter what source type is.
2. in explode_numbers, the argument column type is certain Int32. so
reduce some virtual function call.
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@924060929@HappenLee
[fix](compile) fix macos compile failed (apache#44061)
7870c3a
fix macos compile failed, introduced by apache#40813, apache#42930, apache#43218, apache#43289
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@HappenLeeHappenLeeHappenLee approved these changes
@MryangeMryangeMryange approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zclllyybb@doris-robot@HappenLee@Mryange@zhangstar333