[fix](tablet io error) fix tablet not increase _io_error_times in some cases. by yujun777 · Pull Request #42208 · apache/doris

yujun777 · 2024-10-21T11:08:15Z

When read/write a tablet, if meet io error, it will increase this tablet's field _io_error_times. And if _io_error_times >= config::max_tablet_io_errors, then be will report this tablet as bad, then fe can drop it later.

But there are a lot of code that may met an io error! And they may forget to increase this tablet's _io_error_times.
This PR only fix some, there should be others not fixed.

What's more, config::max_tablet_io_errors of be.conf default value is -1. It means always no set tablet as bad. For an online cluster, it shoud manually set this parameter > 0 in order to make it work.

doris-robot · 2024-10-21T11:08:21Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions

clang-tidy made some suggestions

github-actions · 2024-10-21T11:13:19Z

+#include "olap/tablet.h"
 #include "olap/tablet_fwd.h"
 #include "olap/tablet_schema.h"
+#include "runtime/exec_env.h"


warning: duplicate include [readability-duplicate-include]
be/src/runtime/load_stream.cpp:39:
- #include "olap/tablet_schema.h"- #include "runtime/exec_env.h"+ #include "olap/tablet_schema.h"

yujun777 · 2024-10-21T11:26:07Z

run buildall

github-actions · 2024-10-21T11:30:40Z

clang-tidy review says "All clean, LGTM! 👍"

yujun777 · 2024-10-21T12:14:51Z

run buildall

doris-robot · 2024-10-21T12:54:16Z

TeamCity be ut coverage result:
Function Coverage: 37.46% (9700/25891)
Line Coverage: 28.71% (80549/280536)
Region Coverage: 28.15% (41680/148059)
Branch Coverage: 24.71% (21179/85706)
Coverage Report: http://coverage.selectdb-in.cc/coverage/58faebf163eeefcf4f6918146056c636b98bbb89_58faebf163eeefcf4f6918146056c636b98bbb89/report/index.html

yujun777 · 2024-10-22T08:09:02Z

run buildall

dataroaring

LGTM

github-actions · 2024-10-22T08:35:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-10-22T08:35:40Z

PR approved by anyone and no changes requested.

yujun777 · 2024-10-22T08:53:06Z

run beut

yujun777 · 2024-10-22T09:48:09Z

run p0

yujun777 · 2024-10-23T03:07:19Z

run buildall

yujun777 · 2024-10-23T03:43:37Z

run buildall

doris-robot · 2024-10-23T04:50:04Z

TeamCity be ut coverage result:
Function Coverage: 37.47% (9710/25912)
Line Coverage: 28.75% (80590/280352)
Region Coverage: 28.16% (41660/147964)
Branch Coverage: 24.71% (21164/85640)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f336a11c51b3632a41a40229c346c77bfa881b42_f336a11c51b3632a41a40229c346c77bfa881b42/report/index.html

yujun777 · 2024-10-23T07:00:56Z

run p0

yujun777 · 2024-10-23T07:21:59Z

run buildall

doris-robot · 2024-10-23T07:47:10Z

TeamCity be ut coverage result:
Function Coverage: 37.49% (9710/25903)
Line Coverage: 28.76% (80593/280262)
Region Coverage: 28.17% (41671/147901)
Branch Coverage: 24.72% (21167/85628)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fdb10b0c99326a608f62c138a3097fd25a8ffb4c_fdb10b0c99326a608f62c138a3097fd25a8ffb4c/report/index.html

dataroaring

LGTM

github-actions · 2024-10-23T07:54:36Z

PR approved by at least one committer and no changes requested.

yujun777 · 2024-10-23T11:10:34Z

run p0

yujun777 · 2024-10-23T11:11:12Z

run p0

yujun777 · 2024-10-23T11:11:22Z

run cloud_p0

deardeng

LGTM

…e cases. (#42208) When read/write a tablet, if meet io error, it will increase this tablet's field `_io_error_times`. And if `_io_error_times` >= config::max_tablet_io_errors, then be will report this tablet as bad, then fe can drop it later. But there are a lot of code that may met an io error! And they may forget to increase this tablet's _io_error_times. This PR only fix some, there should be others not fixed. What's more, config::max_tablet_io_errors of be.conf default value is -1. It means always no set tablet as bad. For an online cluster, it shoud manually set this parameter > 0 in order to make it work.

…times in some cases. (#42955) PR Body: When read/write a tablet, if meet io error, it will increase this tablet's field `_io_error_times`. And if `_io_error_times` >= config::max_tablet_io_errors, then be will report this tablet as bad, then fe can drop it later. But there are a lot of code that may met an io error! And they may forget to increase this tablet's _io_error_times. This PR only fix some, there should be others not fixed. What's more, config::max_tablet_io_errors of be.conf default value is -1. It means always no set tablet as bad. For an online cluster, it shoud manually set this parameter > 0 in order to make it work. Cherry-picked from #42208 Co-authored-by: yujun <yu.jun.reach@gmail.com>

…7561) Cherry-pick all docker/runtime/ changes from apache/doris master that are missing in branch-hotfix-selectdb-doris-3.1-hubspot. Includes changes from PRs: apache#42068 apache#42208 apache#42564 apache#42605 apache#43040 apache#43085 apache#43200 apache#45503 apache#45926 apache#46387 apache#47342 apache#47837 apache#48183 apache#48839 apache#48898 apache#49343 apache#49874 apache#50247 apache#50536 apache#50604 apache#50745 apache#50922 apache#51357 apache#51693 apache#54321 apache#54392 apache#54950 apache#55299 apache#55548 apache#55615 apache#55671 apache#55824 apache#56471 apache#56757 apache#56917 apache#57113 apache#57236 apache#57345 apache#57374 apache#57695 apache#58229 apache#58565 apache#58638 apache#59039 apache#59292 apache#59480 apache#59553 apache#60060 apache#60458 ## Proposed changes Issue Number: close #xxx  Co-authored-by: catpineapple <catpineapple@users.noreply.github.com>

…e cases. (apache#42208) When read/write a tablet, if meet io error, it will increase this tablet's field `_io_error_times`. And if `_io_error_times` >= config::max_tablet_io_errors, then be will report this tablet as bad, then fe can drop it later. But there are a lot of code that may met an io error! And they may forget to increase this tablet's _io_error_times. This PR only fix some, there should be others not fixed. What's more, config::max_tablet_io_errors of be.conf default value is -1. It means always no set tablet as bad. For an online cluster, it shoud manually set this parameter > 0 in order to make it work.

github-actionsBot reviewed Oct 21, 2024
View reviewed changes

yujun777 force-pushed the test-22 branch 2 times, most recently from 58faebf to 4c0cb09Compare October 21, 2024 11:26

yujun777 force-pushed the test-22 branch from 4c0cb09 to 58faebfCompare October 21, 2024 12:14

yujun777 force-pushed the test-22 branch from 58faebf to 56c6b69Compare October 22, 2024 08:08

dataroaring previously approved these changes Oct 22, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 22, 2024

yujun777 dismissed dataroaring’s stale review via ce2d39fOctober 23, 2024 03:07

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Oct 23, 2024

yujun777 added 3 commits October 23, 2024 15:21

fix tablet io error not set bad
79c1c0c

fix be ut
c44566a

fix be ut
fdb10b0

yujun777 force-pushed the test-22 branch from f336a11 to fdb10b0Compare October 23, 2024 07:21

dataroaring approved these changes Oct 23, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 23, 2024

deardeng approved these changes Oct 30, 2024
View reviewed changes

dataroaring added dev/2.1.x dev/3.0.x and removed dev/2.1.x labels Oct 31, 2024

dataroaring merged commit f5642d2 into apache:masterOct 31, 2024

github-actionsBot mentioned this pull request Oct 31, 2024
branch-3.0: [fix](tablet io error) fix tablet not increase _io_error_times in some cases. #42955
Merged

dataroaring added dev/3.0.3-merged and removed dev/3.0.x labels Nov 7, 2024

Uh oh!

Conversation

yujun777 commented Oct 21, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Oct 21, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBotOct 21, 2024

Choose a reason for hiding this comment

Uh oh!

yujun777 commented Oct 21, 2024

Uh oh!

github-actionsBot commented Oct 21, 2024

Uh oh!

yujun777 commented Oct 21, 2024

Uh oh!

doris-robot commented Oct 21, 2024

Uh oh!

yujun777 commented Oct 22, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 22, 2024

Uh oh!

github-actionsBot commented Oct 22, 2024

Uh oh!

yujun777 commented Oct 22, 2024

Uh oh!

yujun777 commented Oct 22, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

doris-robot commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

doris-robot commented Oct 23, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

yujun777 commented Oct 23, 2024

Uh oh!

deardeng left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yujun777 commented Oct 21, 2024•
edited
Loading