Skip to content

[Enhance](ip)optimize priority_ network matching logic for be - #23795

Merged
yiguolei merged 11 commits into
apache:masterfrom
zddr:pn_be
Sep 11, 2023
Merged

[Enhance](ip)optimize priority_ network matching logic for be#23795
yiguolei merged 11 commits into
apache:masterfrom
zddr:pn_be

Conversation

@zddr

@zddrzddr commented Sep 3, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

  • If the user has configured the wrong priority_network, direct startup failure to avoid users mistakenly assuming that the configuration is correct
  • If the user has not configured p_ n. Select only the first IP from the IPv4 list, rather than selecting from all IPs, to avoid users' servers not supporting IPv4

extends #23784

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@zddr

zddr commented Sep 3, 2023

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Sep 3, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@hello-stephen

hello-stephen commented Sep 3, 2023

Copy link
Copy Markdown
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.92 seconds
stream load tsv: 529 seconds loaded 74807831229 Bytes, about 134 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162197453 Bytes

@dataroaring

dataroaring commented Sep 3, 2023

Copy link
Copy Markdown
Contributor

please describe changes in commit message and add ut if possible.

github-actions[bot]
github-actionsBot reviewed Sep 8, 2023
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
@zddr

zddr commented Sep 8, 2023

Copy link
Copy Markdown
ContributorAuthor

run beut

@doris-robot

doris-robot commented Sep 8, 2023

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.92% (7896/21388)
Line Coverage: 28.92% (63354/219040)
Region Coverage: 27.87% (32898/118052)
Branch Coverage: 24.45% (16894/69096)
Coverage Report: http://coverage.selectdb-in.cc/coverage/37e3b82a9002eea294ac59a47e6c4e3997f1b6a2_37e3b82a9002eea294ac59a47e6c4e3997f1b6a2/report/index.html

@zddr

zddr commented Sep 8, 2023

Copy link
Copy Markdown
ContributorAuthor

run beut

github-actions[bot]
github-actionsBot reviewed Sep 8, 2023
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp
Comment threadbe/test/util/backend_options_test.cpp Outdated
Comment threadbe/test/util/backend_options_test.cpp Outdated
Comment threadbe/test/util/backend_options_test.cpp Outdated
Comment threadbe/test/util/backend_options_test.cpp Outdated
Comment threadbe/test/util/backend_options_test.cpp Outdated
@zddr

zddr commented Sep 8, 2023

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Sep 8, 2023

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.93% (7898/21389)
Line Coverage: 28.94% (63390/219049)
Region Coverage: 27.88% (32914/118043)
Branch Coverage: 24.47% (16905/69096)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cb7c6672dfabefde48b09f311f47d0236d0b2080_cb7c6672dfabefde48b09f311f47d0236d0b2080/report/index.html

@doris-robot

doris-robot commented Sep 8, 2023

Copy link
Copy Markdown

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.15 seconds
stream load tsv: 533 seconds loaded 74807831229 Bytes, about 133 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.8 seconds inserted 10000000 Rows, about 335K ops/s
storage size: 17162220188 Bytes

morningman
morningman previously approved these changes Sep 10, 2023
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2023
@github-actions

github-actionsBot commented Sep 10, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Sep 10, 2023

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Sep 10, 2023
@zddr

zddr commented Sep 10, 2023

Copy link
Copy Markdown
ContributorAuthor

run buildall

morningman
morningman approved these changes Sep 10, 2023
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2023
@github-actions

github-actionsBot commented Sep 10, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot

doris-robot commented Sep 10, 2023

Copy link
Copy Markdown

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.3 seconds
stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162211266 Bytes

@zddr

zddr commented Sep 11, 2023

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Sep 11, 2023

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.93% (7915/21434)
Line Coverage: 28.96% (63603/219633)
Region Coverage: 27.88% (32986/118330)
Branch Coverage: 24.45% (16937/69268)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ab79476b329cc1ffe8b1325659902a1963f54a43_ab79476b329cc1ffe8b1325659902a1963f54a43/report/index.html

@doris-robot

doris-robot commented Sep 11, 2023

Copy link
Copy Markdown

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.62 seconds
stream load tsv: 586 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162172744 Bytes

hf200012
hf200012 approved these changes Sep 11, 2023
View reviewed changes

@hf200012hf200012 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
yiguolei merged commit dbb9365 into apache:masterSep 11, 2023
xiaokang pushed a commit that referenced this pull request Sep 11, 2023
@zddr@xiaokang
[Enhance](ip)optimize priority_ network matching logic for be (#23795)
e8aa24a
Issue Number: close #xxx
If the user has configured the wrong priority_network, direct startup failure to avoid users mistakenly assuming that the configuration is correct
If the user has not configured p_ n. Select only the first IP from the IPv4 list, rather than selecting from all IPs, to avoid users' servers not supporting IPv4
extends #23784
xiaokang pushed a commit that referenced this pull request Sep 13, 2023
@zddr@xiaokang
[Enhance](ip)optimize priority_ network matching logic for be (#23795)
e841bb1
Issue Number: close #xxx
If the user has configured the wrong priority_network, direct startup failure to avoid users mistakenly assuming that the configuration is correct
If the user has not configured p_ n. Select only the first IP from the IPv4 list, rather than selecting from all IPs, to avoid users' servers not supporting IPv4
extends #23784
@xiaokangxiaokang mentioned this pull request Sep 28, 2023
Closed
@xiaokangxiaokang mentioned this pull request Dec 3, 2023
Closed
@zddr
zddr deleted the pn_be branch March 28, 2024 02:32
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@hf200012hf200012hf200012 approved these changes
@morningmanmorningmanmorningman approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.0.2-mergedkind/behavior-changedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@zddr@hello-stephen@dataroaring@doris-robot@hf200012@morningman@xiaokang@yiguolei