Skip to content

[fix](brokerload) fix be core dump casued by broker load - #15874

Merged
yangzhg merged 1 commit into
apache:masterfrom
luozenglin:broker_load_fix
Jan 17, 2023
Merged

[fix](brokerload) fix be core dump casued by broker load#15874
yangzhg merged 1 commit into
apache:masterfrom
luozenglin:broker_load_fix

Conversation

@luozenglin

@luozenglinluozenglin commented Jan 12, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close#15846

Problem summary

The BrokerFileReader is refactored in #15622, but the _client is not initialized

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions

github-actionsBot commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

dataroaring
dataroaring previously approved these changes Jan 12, 2023
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring

dataroaring commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

We should add regression test for broker load.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Jan 12, 2023
@github-actions

github-actionsBot commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

hello-stephen commented Jan 12, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.71 seconds
load time: 503 seconds
storage size: 17122332084 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230116194451_clickbench_pr_81624.html

HappenLee
HappenLee reviewed Jan 13, 2023
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not call ·_fs->get_client in BrokerFileReader Contructor ?

@luozenglin

luozenglin commented Jan 13, 2023

Copy link
Copy Markdown
ContributorAuthor

why not call ·_fs->get_client in BrokerFileReader Contructor ?

The BrokerFileReader may be constructed before connect(), when the client is not yet constructed.

@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jan 13, 2023
@github-actions

github-actionsBot commented Jan 13, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@luozenglin
luozenglinforce-pushed the broker_load_fix branch 2 times, most recently from 76e0680 to 068a4ceCompareJanuary 16, 2023 02:40
@github-actions

github-actionsBot commented Jan 16, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions

github-actionsBot commented Jan 16, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions

github-actionsBot commented Jan 16, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions

github-actionsBot commented Jan 16, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

yangzhg
yangzhg approved these changes Jan 17, 2023
View reviewed changes

@yangzhgyangzhg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yangzhg
yangzhg merged commit 9755358 into apache:masterJan 17, 2023
@ByteYueByteYue mentioned this pull request Jan 17, 2023
Closed
13 tasks
@luozenglin

luozenglin commented Jan 17, 2023

Copy link
Copy Markdown
ContributorAuthor

why not call ·_fs->get_client in BrokerFileReader Contructor ?

The BrokerFileReader may be constructed before connect(), when the client is not yet constructed.

Sorry, I made a mistake in my reply above. There is no problem getting the client in the constructor and the logic is much clearer.

@luozenglinluozenglin mentioned this pull request Jan 17, 2023
Merged
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLeeHappenLeeHappenLee left review comments
@yangzhgyangzhgyangzhg approved these changes
@dataroaringdataroaringdataroaring left review comments

Assignees

No one assigned

Labels

reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] [daily] be core when load from hdfs

5 participants

@luozenglin@dataroaring@hello-stephen@yangzhg@HappenLee