Skip to content
This repository was archived by the owner on Jul 24, 2024. It is now read-only.

log_restore: refactor write and ingest logic. - #645

Merged
kennytm merged 63 commits into
pingcap:masterfrom
3pointer:refactor_ingest
Jan 31, 2021
Merged

log_restore: refactor write and ingest logic.#645
kennytm merged 63 commits into
pingcap:masterfrom
3pointer:refactor_ingest

Conversation

@3pointer

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Both Lightning and BR have a similar logic. write and ingest kv as well as retry policy. This abstract Ingest method for BR and Lightning easy to use.

What is changed and how it works?

  1. Refactor write and ingest code from log_client to Ingestser.

Check List

Tests

  • No code

Related changes

  • Need to cherry-pick to the release branch

Release Note

  • No release note.

@3pointer

Copy link
Copy Markdown
CollaboratorAuthor

/run-integration-test

Comment threadpkg/kv/kv.go Outdated
Comment threadpkg/kv/kv.go Outdated
Comment threadpkg/kv/kv.go
Comment threadpkg/restore/ingester.go Outdated
Comment threadpkg/restore/ingester.go Outdated
}
switch retryTy {
case retryNone:
log.L().Warn("ingest failed and do not retry", zap.Error(err), zap.Reflect("meta", meta),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use log.ShortError instead

Comment threadpkg/restore/range.go Outdated

@glorvglorv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

Comment threadpkg/kv/kv.go Outdated
Comment threadpkg/kv/kv.go Outdated
// Error return current error on this iter.
Error() error
// First return the first key in this iter.
First() []byte

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
First() []byte
First() bool

Return a bool to indicate if the key is valid

Comment threadpkg/kv/kv.go Outdated
// First return the first key in this iter.
First() []byte
// Last return the last key in this iter.
Last() []byte

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

3pointerand others added 2 commits January 12, 2021 19:53
Co-authored-by: glorv <glorvs@163.com>
@glorv

Copy link
Copy Markdown
Collaborator

LGTM

Comment threadpkg/kv/kv.go Outdated
Comment threadpkg/kv/kv.go Outdated
Comment threadpkg/restore/range_test.go Outdated
// Copyright 2020 PingCAP, Inc. Licensed under Apache-2.0.

package restore_test
package restore

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

????

Comment threadpkg/restore/split_client.go Outdated
@3pointer

Copy link
Copy Markdown
CollaboratorAuthor

/run-all-tests

@kennytm

Copy link
Copy Markdown
Collaborator

/lgtm

@ti-srebotti-srebot removed the status/LGT1 LGTM1 label Jan 29, 2021
@kennytm
kennytm merged commit d87292c into pingcap:masterJan 31, 2021
ti-srebot pushed a commit to ti-srebot/br that referenced this pull request Jan 31, 2021
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot

Copy link
Copy Markdown
Contributor

cherry pick to release-4.0 in PR #729

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@3pointer@glorv@kennytm@ti-srebot@overvenus