Skip to content

[fix](compatibility) Fix compatibility issue of PRowBatch and some tablet sink bugs - #8000

Merged
morningman merged 8 commits into
apache:masterfrom
morningman:tuple_offsets_compatible
Feb 15, 2022
Merged

[fix](compatibility) Fix compatibility issue of PRowBatch and some tablet sink bugs#8000
morningman merged 8 commits into
apache:masterfrom
morningman:tuple_offsets_compatible

Conversation

@morningman

@morningmanmorningman commented Feb 9, 2022

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

Problem Summary:

  1. set both tuple_offsets and new_tuple_offsets in PRowBatch for compatibility
  2. set FE config repair_slow_replica default to false
    Avoid impacting the load process after upgrading.
    Eg, if there are only 2 replicas, one is with high version count. After upgrade,
    that replica will be set to bad, so that the load process will be stopped
    because only 1 replica is alive.
  3. Fix a bug that NodeChannel may be blocked at close_wait()
    Forget to set add_batch_finish flag after the last rpc finished.
  4. Fix a NPE of RoutineLoadScheduler

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morningmanmorningman added the kind/fix Categorizes issue or PR as related to a bug. label Feb 9, 2022
@morningmanmorningman changed the title [fix](compatibility) Fix compatibility issue of PRowBatch[fix](compatibility) Fix compatibility issue of PRowBatch and some tablet sink bugsFeb 10, 2022
Comment threadbe/src/exec/tablet_sink.cpp Outdated
auto st = channel->add_row(tuple, tablet_id);
if (!st.ok()) {
mark_as_failed(channel, st.get_error_msg(), tablet_id);
mark_as_failed(channel.get(), st.get_error_msg(), tablet_id);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not change mark_as_failed signature to accept shared_ptr ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is not necessary, and it will cause extra shared ptr copy.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I don't think it's a good practice to mix smart pointers and raw pointers. When we use smart pointers, we already imply that object ownership is managed by smart pointers instead of accessing raw pointers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I will change it

yangzhg
yangzhg previously approved these changes Feb 12, 2022

@yangzhgyangzhg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 12, 2022
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

morningman-cmyand others added 7 commits February 14, 2022 09:31
1. set both tuple_offsets and new_tuple_offsets in PRowBatch for compatibility
2. set FE config `repair_slow_replica` default to false
Avoid impacting the load process after upgrading.
Eg, if there are only 2 replicas, one is with high version count. After upgrade,
that replica will be set to bad, so that the load process will be stopped
because only 1 replica is alive.
@morningman
morningmanforce-pushed the tuple_offsets_compatible branch from 86c5ca8 to fb21ee0CompareFebruary 14, 2022 01:53
@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Feb 14, 2022

@yangzhgyangzhg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 14, 2022
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@morningman
morningman merged commit 884fddb into apache:masterFeb 15, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.kind/fixCategorizes issue or PR as related to a bug.reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@morningman@yangzhg@morningman-cmy