Uh oh!
There was an error while loading. Please reload this page.
[HBASE-25536] Remove 0 length wal file from logQueue if it belongs to… - #2908
Merged
Conversation
Apache-HBase
commented
Jan 28, 2021
🎊 +1 overall
This message was automatically generated. |
wchevreuil
left a comment
Contributor
There was a problem hiding this comment.
Makes sense to me. How difficult is it to add UTs testing this?
| private void handleEofException(IOException e) { | ||
| if ((e instanceof EOFException || e.getCause() instanceof EOFException) && | ||
| logQueue.size() > 1 && this.eofAutoRecovery) { | ||
| (source.isRecovered() || logQueue.size() > 1) && this.eofAutoRecovery) { |
Contributor
There was a problem hiding this comment.
nice find, mind adding a quick comment since this is a subtle behavior?
Contributor
There was a problem hiding this comment.
On a related note, should we add a source and global metric counter to track the number of 0 size files dequeued? Seems more common than we thought, easy to monitor may be..
ContributorAuthor
There was a problem hiding this comment.
On a related note, should we add a source and global metric counter to track the number of 0 size files dequeued? Seems more common than we thought, easy to monitor may be..
@bharathv Good idea. Could you please create a separate jira to track this. Thank you !
Apache-HBase
commented
Jan 28, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jan 28, 2021
🎊 +1 overall
This message was automatically generated. |
shahrs87
commented
Jan 28, 2021
ContributorAuthor
Will add in next commit. |
Apache-HBase
commented
Jan 28, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jan 28, 2021
💔 -1 overall
This message was automatically generated. |
shahrs87
commented
Jan 28, 2021
ContributorAuthor
This patch applies cleanly for branch-2 |
bharathv
approved these changes
Jan 29, 2021
Apache-HBase
commented
Jan 29, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jan 29, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jan 29, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jan 29, 2021
💔 -1 overall
This message was automatically generated. |
virajjasani
left a comment
Contributor
There was a problem hiding this comment.
Left small nit, looks good otherwise
| when(source.isPeerEnabled()).thenReturn(true); | ||
| ReplicationSourceWALReader reader = | ||
| new ReplicationSourceWALReader(fs, conf, queue, 0, getDummyFilter(), source); | ||
| reader.run(); |
virajjasani
approved these changes
Jan 29, 2021
| reader.run(); | ||
| // ReplicationSourceWALReaderThread#handleEofException method will | ||
| // remove empty log from logQueue. | ||
| assertEquals(0, queue.size()); |
Contributor
There was a problem hiding this comment.
I see, If we use thread.start(), this could be flaky if we reach here before reader thread removes entry from queue so we will need to add some wait here based on this exit condition (queue.isEmpty()). It's a trade off with starting thread separately as opposed to synchronously executing it with thread.run().
It's upto you, this is anyways trivial. thread.run() is also fine.
virajjasani pushed a commit
that referenced
this pull request
Jan 29, 2021
…ld sources (#2908) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
virajjasani pushed a commit
that referenced
this pull request
Jan 29, 2021
…ld sources (#2908) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
virajjasani pushed a commit
that referenced
this pull request
Jan 29, 2021
…ld sources (#2908) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
virajjasani pushed a commit
that referenced
this pull request
Jan 29, 2021
…ld sources (#2908) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
wchevreuil pushed a commit
to wchevreuil/hbase
that referenced
this pull request
May 24, 2021
…ld sources (apache#2908) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> (cherry picked from commit ea1a9e9) Change-Id: Ifb5a34850ff75f4353fc1f0aa6f534897bd5458a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… old sources.