Skip to content

branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179) - #62961

Merged
yiguolei merged 1 commit into
branch-4.1from
io_helper_41
May 1, 2026
Merged

branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)#62961
yiguolei merged 1 commit into
branch-4.1from
io_helper_41

Conversation

@zclllyybb

Copy link
Copy Markdown
Contributor

pick #62179

CopilotAI review requested due to automatic review settings April 30, 2026 03:07
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zclllyybbzclllyybb changed the title [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)Apr 30, 2026
@zclllyybb

Copy link
Copy Markdown
ContributorAuthor

run buildall

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors Doris BE casting/serde/conversion code to remove the legacy util/io_helper indirection and route parsing/formatting through the canonical CastTo* implementations.

Changes:

  • Delete be/src/util/io_helper.{h,cpp} and replace call sites with CastToInt/CastToFloat/CastToDecimal/CastToString and datelike cast implementations.
  • Extend cast helpers (notably CastToString 128-bit overloads) and update converters/serdes to use the new entry points.
  • Add/extend BE unit tests covering 128-bit string formatting, DecimalV2 cast behavior, datelike conversions, Arrow parsing, and sequence-match time conditions.

Reviewed changes

Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.

Show a summary per file
FileDescription
be/test/exprs/function/cast/cast_to_string_api_test.cppAdds tests for new CastToString 128-bit overload behavior.
be/test/exprs/function/cast/cast_to_float_double.cppAdds regression test ensuring DecimalV2 preserves scale when casting to float/double.
be/test/exprs/aggregate/vec_sequence_match_test.cppAdds tests for time-condition parsing + serialize/deserialize and malformed numeric conditions.
be/test/exec/column_type_convert_test.cppUpdates largeint formatting to CastToString and adds string→datelike conversion tests.
be/test/core/field_test.cppReplaces io_helper include with CastToString.
be/test/core/data_type_serde/data_type_serde_number_test.cppAdds Arrow string→(DateV2/DateTimeV2) numeric parsing coverage.
be/test/core/data_type_serde/data_type_serde_decimal_test.cppAdds JSON deserialize test to preserve underflow compatibility.
be/test/common/check_overflow.cppSwitches integer parsing from io_helper helpers to CastToInt.
be/src/util/jsonb_document_cast.hRoutes decimal→decimal casting via CastToDecimal::from_decimal.
be/src/util/io_helper.hRemoves legacy helper header.
be/src/util/io_helper.cppRemoves legacy helper implementation.
be/src/util/counts.hDrops io_helper include.
be/src/storage/olap_scan_common.hDrops io_helper include.
be/src/format/column_type_convert.hReplaces io_helper parsing with CastTo* APIs and threads CastParameters through string-cast helpers.
be/src/exprs/function/function_json.cppDrops io_helper include.
be/src/exprs/function/cast/cast_to_timestamptz.hDrops io_helper include.
be/src/exprs/function/cast/cast_to_string.hAdds explicit from_int128/from_uint128 helpers and wires them into from_number.
be/src/exprs/function/cast/cast_to_jsonb.hDrops io_helper include.
be/src/exprs/function/cast/cast_to_int.hUses CastToInt::from_decimal and simplifies decimal overflow/nullability handling.
be/src/exprs/function/cast/cast_to_float.hUses CastToFloat::from_decimal for decimal→float/double conversions.
be/src/exprs/function/cast/cast_to_decimal.hReplaces io_helper decimal parsing and introduces from_decimal wrapper for decimal→decimal casts.
be/src/exprs/function/cast/cast_to_boolean.hReplaces bool parsing with StringParser::string_to_bool.
be/src/exprs/function/cast/cast_to_basic_number_common.hCentralizes numeric parsing via StringParser and provides shared int/float cast helpers.
be/src/exprs/function/cast/cast_base.hDrops io_helper include from cast base utilities.
be/src/exprs/aggregate/moments.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_uniq.hDrops io_helper include; adds needed column/string headers.
be/src/exprs/aggregate/aggregate_function_topn.hDrops io_helper include; adds needed column/string headers.
be/src/exprs/aggregate/aggregate_function_sequence_match.hReplaces integer parsing with StringParser, improves parse-failure reset behavior.
be/src/exprs/aggregate/aggregate_function_rpc.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_product.hAdjusts debug logging to format int128_t via CastToString.
be/src/exprs/aggregate/aggregate_function_min_max.hDrops io_helper include; adds decimal column header.
be/src/exprs/aggregate/aggregate_function_java_udaf.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_histogram.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_corr.hDrops io_helper include; adds standard headers required by implementation.
be/src/exprs/aggregate/aggregate_function_collect.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_bit.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_avg_weighted.hDrops io_helper include.
be/src/exprs/aggregate/aggregate_function_array_agg.hDrops io_helper include.
be/src/exec/common/histogram_helpers.hppDrops io_helper include.
be/src/exec/common/hash_table/hash_table.hDrops io_helper include.
be/src/core/field.cppDrops io_helper include.
be/src/core/data_type_serde/data_type_serde.hUpdates comment to reference CastToDecimal::from_string.
be/src/core/data_type_serde/data_type_number_serde.cppSwitches parsing/formatting to CastTo* APIs (incl. IPv6 formatting).
be/src/core/data_type_serde/data_type_ipv6_serde.cppUses CastToIPv6::from_string instead of io_helper reader.
be/src/core/data_type_serde/data_type_ipv4_serde.cppUses CastToIPv4::from_string instead of io_helper reader.
be/src/core/data_type_serde/data_type_decimal_serde.cppReplaces decimal JSON parsing with StringParser-based paths.
be/src/core/data_type_serde/data_type_datev2_serde.cppUses CastToDateV2 parsing instead of io_helper reader.
be/src/core/data_type_serde/data_type_datetimev2_serde.cppUses CastToDatetimeV2 parsing instead of io_helper reader.
be/src/core/data_type_serde/data_type_date_or_datetime_serde.cppUses CastToDateOrDatetime parsing and explicit date/datetime post-processing.
be/src/core/data_type/data_type_quantilestate.cppDrops io_helper include.
be/src/core/data_type/data_type_number_base.cppUses CastToString for 128-bit number formatting.
be/src/core/data_type/data_type_ipv6.cppDrops io_helper include.
be/src/core/data_type/data_type_ipv4.cppDrops io_helper include.
be/src/core/data_type/data_type_hll.cppDrops io_helper include.
be/src/core/data_type/data_type_decimal.cppDrops io_helper include.
be/src/core/data_type/data_type_date_time.cppDrops io_helper include.
be/src/core/data_type/data_type_date_or_datetime_v2.cppDrops io_helper include.
be/src/core/data_type/data_type_date.cppDrops io_helper include.
be/src/core/data_type/data_type_bitmap.cppDrops io_helper include.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +211 to +218
StringParser::ParseResult result = StringParser::PARSE_SUCCESS;
if constexpr (T == TYPE_DECIMALV2) {
val = DecimalV2Value(StringParser::string_to_decimal<TYPE_DECIMALV2>(
str_ref.data, cast_set<Int32>(str_ref.size), DecimalV2Value::PRECISION,
DecimalV2Value::SCALE, &result));
} else {
val.value = StringParser::string_to_decimal<T>(str_ref.data, cast_set<Int32>(str_ref.size),
precision, scale, &result);
Comment on lines 41 to 44
#include "exprs/function/cast/cast_parameters.h"
#include "exprs/function/function.h"
#include "exprs/function/function_helpers.h"
#include "util/io_helper.h"

@zclllyybb

Copy link
Copy Markdown
ContributorAuthor

run buildall

…#62179)
Problem Summary: Remove the legacy io_helper wrapper layer and move its
parsing and formatting entry points onto the existing cast
implementations so serde and conversion code depends on the real owners
of that behavior.
@zclllyybb

Copy link
Copy Markdown
ContributorAuthor

run buildall

@yiguolei
yiguolei merged commit f632c93 into branch-4.1May 1, 2026
28 of 32 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@zclllyybb@hello-stephen@yiguolei