[fix](Backup) use correct variable name to check the conf correctness when doing backup job by ByteYue · Pull Request #25434 · apache/doris

ByteYue · 2023-10-13T11:56:19Z

Proposed changes

Issue Number: close #xxx
When using backup on doris, user might encounter following error

We need to use the correct variable name passed from FE like what are defined;

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

ByteYue · 2023-10-13T11:57:07Z

run buildall

github-actions · 2023-10-13T12:04:06Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2023-10-13T12:25:36Z

TeamCity be ut coverage result:
Function Coverage: 36.27% (8144/22454)
Line Coverage: 28.50% (65288/229117)
Region Coverage: 27.18% (33810/124411)
Branch Coverage: 23.98% (17250/71944)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7b6c515388e00f4d86b3f85258982022333e79fd_7b6c515388e00f4d86b3f85258982022333e79fd/report/index.html

doris-robot · 2023-10-13T12:38:09Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.94 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162195075 Bytes

ByteYue · 2023-10-14T12:37:56Z

run buildall

github-actions · 2023-10-14T12:47:24Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2023-10-14T13:06:53Z

TeamCity be ut coverage result:
Function Coverage: 36.27% (8145/22456)
Line Coverage: 28.50% (65299/229133)
Region Coverage: 27.19% (33824/124421)
Branch Coverage: 23.99% (17258/71952)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d3272945cbf8852895c8d61446cd8176ce2b8939_d3272945cbf8852895c8d61446cd8176ce2b8939/report/index.html

doris-robot · 2023-10-14T13:20:58Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.08 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162095659 Bytes

ByteYue · 2023-10-15T10:11:31Z

run buildall

github-actions · 2023-10-15T10:20:48Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2023-10-15T10:33:32Z

TeamCity be ut coverage result:
Function Coverage: 36.64% (8228/22456)
Line Coverage: 28.78% (65933/229132)
Region Coverage: 27.48% (34197/124422)
Branch Coverage: 24.20% (17410/71952)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ecf683e7f9ece04b77b7cd1203ffe21dcebbdb8c_ecf683e7f9ece04b77b7cd1203ffe21dcebbdb8c/report/index.html

doris-robot · 2023-10-15T10:54:32Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.5 seconds
stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162372173 Bytes

ByteYue · 2023-10-16T02:32:20Z

run buildall

github-actions · 2023-10-16T02:40:35Z

clang-tidy review says "All clean, LGTM! 👍"

ByteYue · 2023-10-16T02:44:40Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2023-10-16T02:52:06Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2023-10-16T02:59:00Z

TeamCity be ut coverage result:
Function Coverage: 36.65% (8230/22457)
Line Coverage: 28.78% (65962/229156)
Region Coverage: 27.49% (34208/124429)
Branch Coverage: 24.20% (17416/71958)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3fcd246569ca436aa22f93936e8d2027f8dd095a_3fcd246569ca436aa22f93936e8d2027f8dd095a/report/index.html

doris-robot · 2023-10-16T03:00:10Z

TeamCity be ut coverage result:
Function Coverage: 36.64% (8228/22457)
Line Coverage: 28.78% (65941/229156)
Region Coverage: 27.49% (34210/124429)
Branch Coverage: 24.19% (17410/71958)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fe674664cd31c5d15534cf9bc98447ed2ce8d30e_fe674664cd31c5d15534cf9bc98447ed2ce8d30e/report/index.html

doris-robot · 2023-10-16T03:11:14Z

TeamCity be ut coverage result:
Function Coverage: 36.64% (8228/22457)
Line Coverage: 28.77% (65939/229155)
Region Coverage: 27.49% (34200/124429)
Branch Coverage: 24.19% (17410/71958)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d758b2c3109b8e524ba80f329522980e53b4de72_d758b2c3109b8e524ba80f329522980e53b4de72/report/index.html

doris-robot · 2023-10-16T03:11:54Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.72 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162176009 Bytes

doris-robot · 2023-10-16T03:15:15Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.13 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162242463 Bytes

doris-robot · 2023-10-16T03:28:05Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.35 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162207969 Bytes

w41ter

LGTM

github-actions · 2023-10-16T06:18:02Z

PR approved by anyone and no changes requested.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

ByteYue · 2023-10-16T06:23:15Z

run buildall

github-actions · 2023-10-16T06:30:57Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2023-10-16T06:31:11Z

clang-tidy review says "All clean, LGTM! 👍"

dataroaring

LGTM

github-actions · 2023-10-16T12:13:03Z

PR approved by at least one committer and no changes requested.

morningman · 2023-10-16T14:23:47Z

Fixed by #25496

ByteYue force-pushed the fix_backup_error branch from 7b6c515 to d327294Compare October 14, 2023 12:37

use correct variable name
ecf683e

ByteYue force-pushed the fix_backup_error branch from d327294 to ecf683eCompare October 15, 2023 10:11

for s property compatibility
3fcd246

use old name first fuck
d758b2c

ByteYue force-pushed the fix_backup_error branch from fe67466 to d758b2cCompare October 16, 2023 02:44

github-actionsBot reviewed Oct 16, 2023
View reviewed changes

Comment threadbe/src/util/s3_util.cpp Outdated

w41ter approved these changes Oct 16, 2023
View reviewed changes

github-actionsBot added the reviewed label Oct 16, 2023

ByteYueand others added 2 commits October 16, 2023 14:21

Merge branch 'master' into fix_backup_error
ca6f8dd

Update be/src/util/s3_util.cpp
53fff46
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

dataroaring approved these changes Oct 16, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 16, 2023

dataroaring added the dev/2.0.3 label Oct 16, 2023

morningman closed this Oct 16, 2023

morningman removed the dev/2.0.3 label Oct 16, 2023

Uh oh!

Conversation

ByteYue commented Oct 13, 2023

Proposed changes

Further comments

Uh oh!

ByteYue commented Oct 13, 2023

Uh oh!

github-actionsBot commented Oct 13, 2023

Uh oh!

doris-robot commented Oct 13, 2023

Uh oh!

doris-robot commented Oct 13, 2023

Uh oh!

ByteYue commented Oct 14, 2023

Uh oh!

github-actionsBot commented Oct 14, 2023

Uh oh!

doris-robot commented Oct 14, 2023

Uh oh!

doris-robot commented Oct 14, 2023

Uh oh!

ByteYue commented Oct 15, 2023

Uh oh!

github-actionsBot commented Oct 15, 2023

Uh oh!

doris-robot commented Oct 15, 2023

Uh oh!

doris-robot commented Oct 15, 2023

Uh oh!

ByteYue commented Oct 16, 2023

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

ByteYue commented Oct 16, 2023

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

doris-robot commented Oct 16, 2023

Uh oh!

w41ter left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

ByteYue commented Oct 16, 2023

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 16, 2023

Uh oh!

morningman commented Oct 16, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants