[bugfix](segmentload) should remove segment from segment cache if load segment failed by yiguolei · Pull Request #41608 · apache/doris

yiguolei · 2024-10-09T09:38:17Z

Proposed changes

If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again.
Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached.

doris-robot · 2024-10-09T09:38:22Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

yiguolei · 2024-10-09T09:39:40Z

run buildall

github-actions · 2024-10-09T09:42:35Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-09T10:12:55Z

TeamCity be ut coverage result:
Function Coverage: 37.27% (9629/25838)
Line Coverage: 28.67% (79881/278594)
Region Coverage: 28.10% (41281/146929)
Branch Coverage: 24.72% (21033/85080)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cbc85af3e8cd0a9920cad2cc23b1e386328091d8_cbc85af3e8cd0a9920cad2cc23b1e386328091d8/report/index.html

yiguolei · 2024-10-09T15:42:37Z

run buildall

github-actions · 2024-10-09T15:49:42Z

clang-tidy review says "All clean, LGTM! 👍"

gavinchou · 2024-10-09T17:00:02Z

+ auto status = [this]() {
+ return _inverted_index_file_reader_open.call(
+ [&] { return _open_inverted_index(); });
+ }();


ditto why not just
Suggested change
auto status = [this]() {
return _inverted_index_file_reader_open.call(
[&] { return_open_inverted_index(); });
}();
auto status = _inverted_index_file_reader_open.call([&] { return_open_inverted_index(); });

gavinchou · 2024-10-09T17:01:12Z

- Defer defer([&]() { _footer_pb.reset(); });
- return _create_column_readers(*_footer_pb);
- });
+ auto status = [this]() {


why not just
auto status = _create_column_readers_once_call.call(...);

doris-robot · 2024-10-09T17:12:47Z

TeamCity be ut coverage result:
Function Coverage: 37.30% (9639/25840)
Line Coverage: 28.68% (79919/278631)
Region Coverage: 28.10% (41331/147077)
Branch Coverage: 24.72% (21044/85144)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5aa513a58e75d289d62edba128976f0265cb5422_5aa513a58e75d289d62edba128976f0265cb5422/report/index.html

…d segment failed

github-actions · 2024-10-10T03:53:00Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-10-10T03:56:36Z

clang-tidy review says "All clean, LGTM! 👍"

yiguolei · 2024-10-10T04:08:26Z

run buildall

github-actions · 2024-10-10T04:09:03Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-10-10T04:20:52Z

clang-tidy review says "All clean, LGTM! 👍"

yiguolei · 2024-10-10T04:20:54Z

run buildall

github-actions · 2024-10-10T04:23:14Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-10-10T04:23:52Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-10T05:12:27Z

TeamCity be ut coverage result:
Function Coverage: 37.24% (9643/25893)
Line Coverage: 28.56% (79991/280122)
Region Coverage: 27.98% (41344/147751)
Branch Coverage: 24.60% (21054/85594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c6456cf28bc87476d0d82505640d04ae80fdec7e_c6456cf28bc87476d0d82505640d04ae80fdec7e/report/index.html

eldenmoon

LGTM

github-actions · 2024-10-10T07:25:35Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-10-10T07:25:38Z

PR approved by anyone and no changes requested.

xiaokang

LGTM

…d segment failed (apache#41608) ## Proposed changes 1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again. 2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached. --------- Co-authored-by: yiguolei <yiguolei@gmail.com>

…d segment failed (#41608) (#41660)

## Proposed changes Issue Number: close #xxx introduced by #41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

…he#41685) ## Proposed changes Issue Number: close #xxx introduced by apache#41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

…) (#41697) ## Proposed changes pick #41685 Issue Number: close #xxx introduced by #41608  ## Proposed changes Issue Number: close #xxx  Co-authored-by: yiguolei <yiguolei@gmail.com>

…d segment failed (apache#41608) ## Proposed changes 1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again. 2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached. --------- Co-authored-by: yiguolei <yiguolei@gmail.com>

…he#41685) ## Proposed changes Issue Number: close #xxx introduced by apache#41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

…d segment failed (apache#41608) ## Proposed changes 1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again. 2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached. --------- Co-authored-by: yiguolei <yiguolei@gmail.com>

…he#41685) ## Proposed changes Issue Number: close #xxx introduced by apache#41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

…d segment failed (apache#41608) ## Proposed changes 1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again. 2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached. --------- Co-authored-by: yiguolei <yiguolei@gmail.com>

…he#41685) ## Proposed changes Issue Number: close #xxx introduced by apache#41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

yiguolei force-pushed the fix_seg_st branch from cbc85af to 5aa513aCompare October 9, 2024 15:42

yiguolei added dev/2.1.x dev/3.0.x labels Oct 9, 2024

gavinchou reviewed Oct 9, 2024
View reviewed changes

[bugfix](segmentload) should remove segment from segment cache if loa…
2a276be
…d segment failed

yiguolei force-pushed the fix_seg_st branch from 5aa513a to 2a276beCompare October 10, 2024 03:48

f
4be11be

f
dfc38e3

Doris-Extras added 3 commits October 10, 2024 12:15

f
5ee96ce

f
d97e8e4

f
c6456cf

yiguolei added the p0_c label Oct 10, 2024

eldenmoon approved these changes Oct 10, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 10, 2024

xiaokang reviewed Oct 10, 2024
View reviewed changes

Comment threadbe/src/olap/segment_loader.cpp

xiaokang approved these changes Oct 10, 2024
View reviewed changes

yiguolei merged commit 8f7e106 into apache:masterOct 10, 2024

yiguolei mentioned this pull request Oct 10, 2024
[bugfix](segmentload) should remove segment from segment cache if load segment failed (#41608) #41660
Merged

yiguolei added a commit that referenced this pull request Oct 10, 2024
[bugfix](segmentload) should remove segment from segment cache if loa…
f2ba1f2
…d segment failed (#41608) (#41660)

wm1581066 added dev/2.1.7-merged and removed dev/2.1.x labels Oct 10, 2024

yiguolei mentioned this pull request Oct 11, 2024
[bugfix](becore) use after free problem when the segment is pop #41685
Merged

yiguolei added a commit that referenced this pull request Oct 11, 2024
[bugfix](becore) use after free problem when the segment is pop (#41685)
1f3465f
## Proposed changes Issue Number: close #xxx introduced by #41608  Co-authored-by: yiguolei <yiguolei@gmail.com>

yiguolei mentioned this pull request Oct 11, 2024
[bugfix](becore) use after free problem when the segment is pop (#41685) #41697
Merged

yiguolei mentioned this pull request Oct 19, 2024
[branch-3.0] pick some bugfix #42128
Merged

yiguolei added dev/3.0.3-merged and removed dev/3.0.x labels Oct 19, 2024

Uh oh!

Conversation

yiguolei commented Oct 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

yiguolei commented Oct 9, 2024

Uh oh!

github-actionsBot commented Oct 9, 2024

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

yiguolei commented Oct 9, 2024

Uh oh!

github-actionsBot commented Oct 9, 2024

Uh oh!

gavinchouOct 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gavinchouOct 9, 2024

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

yiguolei commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

yiguolei commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

doris-robot commented Oct 10, 2024

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

github-actionsBot commented Oct 10, 2024

Uh oh!

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

yiguolei commented Oct 9, 2024•
edited
Loading

gavinchouOct 9, 2024•
edited
Loading