Skip to content

[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from - #43624

Merged
yiguolei merged 3 commits into
apache:masterfrom
jacktengg:fix-1108
Nov 13, 2024
Merged

[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from#43624
yiguolei merged 3 commits into
apache:masterfrom
jacktengg:fix-1108

Conversation

@jacktengg

@jacktenggjacktengg commented Nov 11, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
In order to avoid the problem of total string length of a ColumnStr<uint32_t> column exceeds the 4G limit, hash join operator call MutableBlock::merge_ignore_overflow to collect build data.

After finishing collecting build data, a ColumnStr<uint32_t> column will be converted to ColumnStr<uint64_t> if the total string length exceeds the 4G limit by calling Block::replace_if_overflow.

When hash join operator ouput join result block, it need to calls ColumnStr<T>::insert_indices_from to fill a string column, which is able to handle both ColumnStr<uint32_t> and ColumnStr<uint64_t>. Every column which may include string sub-columns should also call insert_indices_from method of it's sub-column to handle both ColumnStr<uint32_t> and ColumnStr<uint64_t>, or else column cast will fail.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot

doris-robot commented Nov 11, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@jacktengg

jacktengg commented Nov 11, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 11, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Nov 11, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg
jacktenggforce-pushed the fix-1108 branch 2 times, most recently from 3241ad5 to 1ebcfc0CompareNovember 11, 2024 16:07
@jacktengg

jacktengg commented Nov 11, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 11, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Nov 11, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg
jacktenggforce-pushed the fix-1108 branch 2 times, most recently from 6fe1301 to d6b0468CompareNovember 12, 2024 07:07
@jacktengg

jacktengg commented Nov 12, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg
jacktenggforce-pushed the fix-1108 branch 2 times, most recently from a0c5a3a to 9482abeCompareNovember 12, 2024 07:43
@jacktengg

jacktengg commented Nov 12, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

yiguolei
yiguolei reviewed Nov 12, 2024
View reviewed changes
Comment threadbe/src/vec/columns/column_string.cpp
yiguolei
yiguolei reviewed Nov 12, 2024
View reviewed changes
Comment threadbe/src/vec/columns/column_string.cpp
yiguolei
yiguolei reviewed Nov 12, 2024
View reviewed changes
Comment threadbe/src/vec/columns/column_string.cpp
yiguolei
yiguolei reviewed Nov 12, 2024
View reviewed changes
Comment threadbe/src/vec/columns/column_string.cpp
@jacktengg

jacktengg commented Nov 12, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 13, 2024
@github-actions

github-actionsBot commented Nov 13, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Nov 13, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Nov 13, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
yiguolei merged commit 0d7b63f into apache:masterNov 13, 2024
github-actionsBot pushed a commit that referenced this pull request Nov 13, 2024
@jacktenggYour Name
[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t…
2ff0ad9
…>>::insert_indices_from (#43624)
@github-actionsgithub-actionsBot mentioned this pull request Nov 13, 2024
Merged
github-actionsBot pushed a commit that referenced this pull request Nov 13, 2024
@jacktenggYour Name
[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t…
91abfd0
…>>::insert_indices_from (#43624)
@github-actionsgithub-actionsBot mentioned this pull request Nov 13, 2024
Merged
zy-kkk pushed a commit that referenced this pull request Nov 13, 2024
@CalvinKirs
[chore](auto-pick)Add the original PR number suffix to the automatica…
3fcd700
…lly picked PR title (#43866)
```
**before**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from
**after**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from #43624
```
yiguolei pushed a commit that referenced this pull request Nov 13, 2024
@github-actions@jacktengg
branch-3.0: [fix](string64) fix coredump caused by ColumnArray<Column…
8febc99
…Str<uint64_t>>::insert_indices_from (#43861)
Cherry-picked from #43624
Co-authored-by: TengJianPing <tengjianping@selectdb.com>
yiguolei pushed a commit that referenced this pull request Nov 13, 2024
@github-actions@jacktengg
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<Column…
d4712ae
…Str<uint64_t>>::insert_indices_from (#43862)
Cherry-picked from #43624
Co-authored-by: TengJianPing <tengjianping@selectdb.com>
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@jacktengg@HappenLee
[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t…
776e93d
…>>::insert_indices_from (apache#43624)
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@CalvinKirs@HappenLee
[chore](auto-pick)Add the original PR number suffix to the automatica…
91ea307
…lly picked PR title (apache#43866)
```
**before**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from
**after**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from apache#43624
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes
@HappenLeeHappenLeeHappenLee approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.8-mergeddev/3.0.3-mergedp0_creviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jacktengg@doris-robot@yiguolei@HappenLee