Skip to content

Ignore broken disks when BE starts up - #3741

Merged
chaoyli merged 1 commit into
apache:masterfrom
spaces-X:diskcheck
Jun 5, 2020
Merged

Ignore broken disks when BE starts up#3741
chaoyli merged 1 commit into
apache:masterfrom
spaces-X:diskcheck

Conversation

@spaces-X

@spaces-Xspaces-X commented Jun 1, 2020

Copy link
Copy Markdown
Contributor

Ignore broken disks from storage_root_path:

  • add read and write check for each dir in storage_root_path and ignore broken disks.
  • add AutoDeallocator.h to manage the lifecycle of allocated data by malloc or posix_memalign.
  • rewrite DataDir::_read_and_write_test_file function.

@morningman

morningman commented Jun 1, 2020

Copy link
Copy Markdown
Contributor

Hi @spaces-X, Could you create an issue to describe what problem you try to solve?

@spaces-X

spaces-X commented Jun 1, 2020

Copy link
Copy Markdown
ContributorAuthor

Hi @spaces-X, Could you create an issue to describe what problem you try to solve?

Done issue

morningman
morningman reviewed Jun 1, 2020
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please update the document of this config ignore_broken_disk, both English and Chinese:

docs/zh-CN/administrator-guide/config/be_config.md
docs/en/administrator-guide/config/be_config.md

Thank you!

Comment threadbe/src/olap/utils.cpp Outdated
@morningmanmorningman added the area/config Issues or PRs related to configuration label Jun 1, 2020
kangkaisen
kangkaisen reviewed Jun 1, 2020
View reviewed changes
Comment threadbe/src/service/doris_main.cpp
kangkaisen
kangkaisen reviewed Jun 1, 2020
View reviewed changes
Comment threadbe/src/util/auto_deallocator.h Outdated
spaces-X
spaces-X commented Jun 2, 2020
View reviewed changes

@spaces-Xspaces-X left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please update the document of this config ignore_broken_disk, both English and Chinese:

docs/zh-CN/administrator-guide/config/be_config.md
docs/en/administrator-guide/config/be_config.md

Thank you!

Done!

Comment threadbe/src/service/doris_main.cpp
kangkaisen
kangkaisen previously approved these changes Jun 2, 2020
View reviewed changes

@kangkaisenkangkaisen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@kangkaisenkangkaisen added the approved Indicates a PR has been approved by one committer. label Jun 2, 2020
@spaces-X

spaces-X commented Jun 3, 2020

Copy link
Copy Markdown
ContributorAuthor

Could I merge this pr to master？
Or what should I do next?

@morningman

morningman commented Jun 3, 2020

Copy link
Copy Markdown
Contributor

Could I merge this pr to master？
Or what should I do next?

Hi, you don't need to do anything. This pr will be merged after being approved for a period of time,
about a day, in case some other reviews have review comments.

@spaces-X

spaces-X commented Jun 3, 2020

Copy link
Copy Markdown
ContributorAuthor

Could I merge this pr to master？
Or what should I do next?

Hi, you don't need to do anything. This pr will be merged after being approved for a period of time,
about a day, in case some other reviews have review comments.

Pull Request for my first time.
Thanks a lot.

chaoyli
chaoyli requested changes Jun 3, 2020
View reviewed changes
Comment threadbe/src/olap/utils.cpp Outdated
@chaoylichaoyli linked an issue Jun 3, 2020 that may be closed by this pull request
Ignore broken disks in the storage_data_path #3746
Closed
@chaoylichaoyli self-assigned this Jun 3, 2020
@spaces-X
spaces-X requested a review from chaoyliJune 3, 2020 12:08
@morningman

morningman commented Jun 3, 2020

Copy link
Copy Markdown
Contributor

Compilation failed. Did you miss util/auto_deallocator.h?

@morningmanmorningman removed the approved Indicates a PR has been approved by one committer. label Jun 3, 2020
@spaces-X

spaces-X commented Jun 3, 2020

Copy link
Copy Markdown
ContributorAuthor

auto_deallocator

sorry, i forgot to remove that line

@spaces-X

spaces-X commented Jun 4, 2020

Copy link
Copy Markdown
ContributorAuthor

Compilation failed. Did you miss util/auto_deallocator.h?

Hi, i have removed the redundant include line, and rebuild it successfully in my machine.
How to handle with the remaining requested changes created by reviewer 'chaoyli', which have already been changed.

@morningman

morningman commented Jun 4, 2020

Copy link
Copy Markdown
Contributor

Hi @chaoyli, Any more comment？

@chaoyli
chaoyli merged commit ed9022a into apache:masterJun 5, 2020
morningman pushed a commit to morningman/doris that referenced this pull request Jun 7, 2020
@spaces-X@morningman-cmy
Ignore broken disk when BE starts up (apache#3741)
86a9950
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman left review comments
@chaoylichaoylichaoyli approved these changes
@kangkaisenkangkaisenkangkaisen left review comments

Assignees

@chaoylichaoyli

Labels

area/configIssues or PRs related to configuration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Ignore broken disks in the storage_data_path

4 participants

@spaces-X@morningman@kangkaisen@chaoyli