HBASE-25541: Setting the path to null when we dequeue the current log by sandeepvinayak · Pull Request #2959 · apache/hbase

sandeepvinayak · 2021-02-16T18:06:49Z

Currently, when we deque we do not reset the current wal path, and it blows up while opening the next log, the current path still points to previous wal which is not correct.

privatebooleanopenNextLog() throwsIOException {
PathnextPath = logQueue.peek();
if (nextPath != null) {
openReader(nextPath); <=== ifweblowuphereif (reader != null) {
returntrue;
}
} else {
// no more files in queue, this could happen for recovered queue, or for a wal group of a sync// replication peer which has already been transited to DA or S.setCurrentPath(null);
}
returnfalse;
}

Apache-HBase · 2021-02-16T18:53:52Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 20s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 51s	master passed
+1 💚	compile	3m 11s	master passed
+1 💚	checkstyle	1m 4s	master passed
+1 💚	spotbugs	2m 3s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 29s	the patch passed
+1 💚	compile	3m 9s	the patch passed
+1 💚	javac	3m 9s	the patch passed
+1 💚	checkstyle	1m 2s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	17m 13s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 7s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		46m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#2959
JIRA Issue	HBASE-25541
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux b7d96ffb323d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b6649a8`
Default Java	AdoptOpenJDK-1.8.0_232-b09
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

bharathv

A nit to add some detail to the commit message to set the context.

bharathv · 2021-02-16T18:58:39Z

 LOG.debug("EOF, closing {}", currentPath);
 closeReader();
 logQueue.remove();
+ setCurrentPath(null);


Nice fix, can you add some detail to the commit message, I think it's subtle.
Looks like the issue is if openNextLog() throws, we want currentPath to be null rather than stale.

I don't have good context here, but is it possible to add a small test? That could help anyone get to know this bit better, it's fine though if it is too complicated to add it.

Got to know from @sandeepvinayak that writing test is complicated. We can skip it.

Apache-HBase · 2021-02-16T20:59:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 47s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 51s	master passed
+1 💚	compile	0m 57s	master passed
+1 💚	shadedjars	6m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 23s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	6m 38s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 36s	the patch passed
		_ Other Tests _
+1 💚	unit	142m 24s	hbase-server in the patch passed.
		172m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#2959
JIRA Issue	HBASE-25541
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 160676e5f9dd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b6649a8`
Default Java	AdoptOpenJDK-1.8.0_232-b09
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/testReport/
Max. process+thread count	4700 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-02-16T21:58:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 54s	master passed
+1 💚	compile	1m 15s	master passed
+1 💚	shadedjars	7m 29s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 39s	the patch passed
+1 💚	compile	1m 14s	the patch passed
+1 💚	javac	1m 14s	the patch passed
+1 💚	shadedjars	7m 28s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	the patch passed
		_ Other Tests _
+1 💚	unit	199m 17s	hbase-server in the patch passed.
		230m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#2959
JIRA Issue	HBASE-25541
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a7f3885b30f4 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b6649a8`
Default Java	AdoptOpenJDK-11.0.6+10
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/testReport/
Max. process+thread count	3265 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

apache#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

apache#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org> (cherry picked from commit ed9f8e6) Change-Id: I6d68df2f91a53fc8eae1a75bc63d847e0da2d63b

Setting the path to null when we dequeue the current log
267c704

bharathv approved these changes Feb 16, 2021
View reviewed changes

virajjasani merged commit 1beda0d into apache:masterFeb 18, 2021

virajjasani pushed a commit that referenced this pull request Feb 18, 2021
HBASE-25541: Setting the path to null when we dequeue the current log (
ec680c5
#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

virajjasani pushed a commit that referenced this pull request Feb 18, 2021
HBASE-25541: Setting the path to null when we dequeue the current log (
f637a60
#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

virajjasani pushed a commit that referenced this pull request Feb 18, 2021
HBASE-25541: Setting the path to null when we dequeue the current log (
1a703f4
#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

virajjasani pushed a commit that referenced this pull request Feb 18, 2021
HBASE-25541: Setting the path to null when we dequeue the current log (
ed9f8e6
#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

virajjasani pushed a commit that referenced this pull request Feb 18, 2021
HBASE-25541: Setting the path to null when we dequeue the current log (
2d26c94
#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

ddupg pushed a commit to ddupg/hbase that referenced this pull request Feb 19, 2021
HBASE-25541 : Setting the path to null when we dequeue the current log (
40e0def
apache#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request Feb 24, 2021
HBASE-25541 : Setting the path to null when we dequeue the current log (
14496c1
apache#2959) Signed-off-by: Bharath Vissapragada <bharathv@apache.org>

Navigation Menu

Uh oh!

HBASE-25541: Setting the path to null when we dequeue the current log - #2959

HBASE-25541: Setting the path to null when we dequeue the current log#2959
virajjasani merged 1 commit into
apache:masterfrom
sandeepvinayak:HBASE-25541

sandeepvinayak commented Feb 16, 2021•
edited
Loading

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

bharathv left a comment

Uh oh!

bharathvFeb 16, 2021

Uh oh!

virajjasaniFeb 17, 2021

Uh oh!

virajjasaniFeb 17, 2021

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

sandeepvinayak commented Feb 16, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

bharathv left a comment

Choose a reason for hiding this comment

Uh oh!

bharathvFeb 16, 2021

Choose a reason for hiding this comment

Uh oh!

virajjasaniFeb 17, 2021

Choose a reason for hiding this comment

Uh oh!

virajjasaniFeb 17, 2021

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

Apache-HBase commented Feb 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sandeepvinayak commented Feb 16, 2021•
edited
Loading