[Refactor](BE) Uniform Expr, Block, Columns size-relative interface (Part I) by zclllyybb · Pull Request #42930 · apache/doris

zclllyybb · 2024-10-30T11:36:05Z

Proposed changes

Issue Number: close #xxx

In this pr we refactored some base interface to avoid narrow cast with data consistent kept.

We uniformed these type of numbers:

columns in Block - U32 (just use no more than U16)
Rows in Block - U64 (others U32, ColumnString64 U64)
column_id in expr arg - I32 (because -1 is meaningful)
child of Expr - U16 (FE use int32, have a restriction of 10000. adjustable but no guarantee for result)

So, for functions' argument column id, we use uint32 to replace size_t.
For rows, we use size_t to replace some int.
For child of expr, we cast it as uint_16 to use, rather than size_t as vector::size()

Refactor of table functions: #43218

doris-robot · 2024-10-30T11:36:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zclllyybb · 2024-10-30T11:39:23Z

run buildall

github-actions

clang-tidy made some suggestions

doris-robot · 2024-10-30T12:39:40Z

TeamCity be ut coverage result:
Function Coverage: 37.89% (9845/25986)
Line Coverage: 29.17% (82001/281131)
Region Coverage: 28.54% (42307/148245)
Branch Coverage: 25.05% (21492/85782)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1cf40cf76bac09d42c7fbb44faf39859eb42cca2_1cf40cf76bac09d42c7fbb44faf39859eb42cca2/report/index.html

github-actions

clang-tidy made some suggestions

zclllyybb · 2024-11-04T08:10:20Z

run buildall

doris-robot · 2024-11-04T09:56:36Z

TeamCity be ut coverage result:
Function Coverage: 37.81% (9821/25975)
Line Coverage: 28.98% (81682/281811)
Region Coverage: 28.23% (42110/149182)
Branch Coverage: 24.81% (21370/86134)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5fdc30f32705d0d9d24d438cd55b0c0b9278da84_5fdc30f32705d0d9d24d438cd55b0c0b9278da84/report/index.html

zclllyybb · 2024-11-05T03:56:35Z

run buildall

doris-robot · 2024-11-05T05:33:04Z

TeamCity be ut coverage result:
Function Coverage: 37.80% (9820/25982)
Line Coverage: 28.96% (81651/281927)
Region Coverage: 28.22% (42106/149227)
Branch Coverage: 24.79% (21360/86158)
Coverage Report: http://coverage.selectdb-in.cc/coverage/44846765b78af1e279faedbd124aa16092f8450f_44846765b78af1e279faedbd124aa16092f8450f/report/index.html

zclllyybb · 2024-11-06T09:42:04Z

run buildall

github-actions

clang-tidy made some suggestions

HappenLee

LGTM

github-actions · 2024-11-06T11:12:43Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-06T11:12:45Z

PR approved by anyone and no changes requested.

doris-robot · 2024-11-06T11:26:12Z

TeamCity be ut coverage result:
Function Coverage: 37.88% (9849/26001)
Line Coverage: 29.04% (81914/282026)
Region Coverage: 28.27% (42194/149229)
Branch Coverage: 24.86% (21410/86130)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c94a3b35e99ed798af16491535e2d1ff61ee8e54_c94a3b35e99ed798af16491535e2d1ff61ee8e54/report/index.html

…Part II) (#43218) part 2 of #42930 two modifications: 1. when parsing data of json array, cast to target type when parse int data no matter what source type is. 2. in explode_numbers, the argument column type is certain Int32. so reduce some virtual function call.

fix macos compile failed, introduced by #40813, #42930, #43218, #43289

fix macos compile failed, introduced by apache#40813, apache#42930, apache#43218, apache#43289 (cherry picked from commit ded2190)

fix macos compile failed, introduced by #40813, #42930, #43218, #43289

…Part I) (apache#42930) ## Proposed changes In this pr we refactored some base interface to avoid narrow cast with data consistent kept. We uniformed these type of numbers: ``` columns in Block - U32 (just use no more than U16) Rows in Block - U64 (others U32, ColumnString64 U64) column_id in expr arg - I32 (because -1 is meaningful) child of Expr - U16 (FE use int32, have a restriction of 10000. adjustable but no guarantee for result) ``` So, for functions' argument column id, we use `uint32` to replace `size_t`. For rows, we use `size_t` to replace some `int`. For child of expr, we cast it as `uint_16` to use, rather than `size_t` as `vector::size()` Refactor of table functions: apache#43218

…Part II) (apache#43218) part 2 of apache#42930 two modifications: 1. when parsing data of json array, cast to target type when parse int data no matter what source type is. 2. in explode_numbers, the argument column type is certain Int32. so reduce some virtual function call.

fix macos compile failed, introduced by apache#40813, apache#42930, apache#43218, apache#43289

zclllyybb force-pushed the compile_check branch from 4221607 to 1cf40cfCompare October 30, 2024 11:39

github-actionsBot reviewed Oct 30, 2024
View reviewed changes

Comment threadbe/src/vec/exprs/table_function/vexplode_json_array.h
Comment threadbe/src/vec/exprs/vcompound_pred.h

Mryange reviewed Oct 31, 2024
View reviewed changes

Comment threadbe/src/vec/exprs/vruntimefilter_wrapper.cpp Outdated