Skip to content

[debug](FileCache) fail over to remote file reader if local cache failed - #24097

Merged
yiguolei merged 1 commit into
apache:masterfrom
AshinGau:file_cache_fail_over
Sep 10, 2023
Merged

[debug](FileCache) fail over to remote file reader if local cache failed#24097
yiguolei merged 1 commit into
apache:masterfrom
AshinGau:file_cache_fail_over

Conversation

@AshinGau

@AshinGauAshinGau commented Sep 8, 2023

Copy link
Copy Markdown
Member

Proposed changes

Fail over to remote file reader even if local file cache failed. This operation can increase the robustness of file cache.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@AshinGau

AshinGau commented Sep 8, 2023

Copy link
Copy Markdown
MemberAuthor

run buildall

@github-actions

github-actionsBot commented Sep 8, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Sep 8, 2023

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.93% (7895/21379)
Line Coverage: 28.93% (63358/218970)
Region Coverage: 27.87% (32897/118017)
Branch Coverage: 24.46% (16896/69070)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8ef368cca51aab60092f0525884cf78a0130d1e8_8ef368cca51aab60092f0525884cf78a0130d1e8/report/index.html

@doris-robot

doris-robot commented Sep 8, 2023

Copy link
Copy Markdown

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.22 seconds
stream load tsv: 532 seconds loaded 74807831229 Bytes, about 134 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162207106 Bytes

morningman
morningman approved these changes Sep 10, 2023
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2023
@github-actions

github-actionsBot commented Sep 10, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Sep 10, 2023

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

yiguolei
yiguolei approved these changes Sep 10, 2023
View reviewed changes

@yiguoleiyiguolei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
yiguolei merged commit 71645a3 into apache:masterSep 10, 2023
xiaokang pushed a commit that referenced this pull request Sep 10, 2023
@AshinGau@xiaokang
[debug](FileCache) fail over to remote file reader if local cache fai…
56d8444
…led (#24097)
Fail over to remote file reader even if local file cache failed. This operation can increase the robustness of file cache.
@xiaokangxiaokang mentioned this pull request Sep 28, 2023
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@yiguoleiyiguoleiyiguolei approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.0.2-mergedp0_breviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@AshinGau@doris-robot@morningman@yiguolei@xiaokang