HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. by comnetwork · Pull Request #1970 · apache/hbase

comnetwork · 2020-06-25T05:43:57Z

In the PR, I added a new method getSyncedLength to WriterBase interface to return the length which successfully synced to underlying fileSystem.
and AbstractFSWAL.getLogFileSizeIfBeingWritten is relied on the added WriteBase.getSyncedLength method:

Apache-HBase · 2020-06-25T05:51:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 15s	Docker failed to build yetus/hbase:023c6fe56b.

Subsystem	Report/Notes
GITHUB PR	#1970
JIRA Issue	HBASE-24625
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/1/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-25T05:51:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 19s	Docker failed to build yetus/hbase:023c6fe56b.

Subsystem	Report/Notes
GITHUB PR	#1970
JIRA Issue	HBASE-24625
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/1/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-25T05:51:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 19s	Docker failed to build yetus/hbase:023c6fe56b.

Subsystem	Report/Notes
GITHUB PR	#1970
JIRA Issue	HBASE-24625
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/1/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

wchevreuil · 2020-06-25T08:51:12Z

retest build

wchevreuil

That is to say, AsyncFSWAL.getLogFileSizeIfBeingWritten could not reflect the file length which successfully synced to underlying HDFS, which is not as expected.

Wasn't that intentional, as a mean to proper track WAL files still open for write? For example, in case of replication, it should go as far as any entry got already appended, no? Ping @Apache9 who worked on this before to give more thoughts.

Apache9 · 2020-06-25T12:30:58Z


 protected FSDataOutputStream output;

+ private volatile long syncedLength = 0;


Use an AtomicLong?

seems that using AtomicLong is unnecessary, because AtomicLong could not provide update if greater than semantics, so I used synchronized keyword here when updating the syncedLength for simplicity

Use AtomicUtils.updateMax. It is a util class in hbase-common.

Used AtomicUtils to replace synchronized, thank you very much.

nit: why do we have AtomicUtils.updateMax? It seems getAndAccumulate is designed for this use case, i.e., syncedLength.getAndAccumulate(fsdos.getPos(), Math::max)

Apache9 · 2020-06-25T12:37:33Z

That is to say, AsyncFSWAL.getLogFileSizeIfBeingWritten could not reflect the file length which successfully synced to underlying HDFS, which is not as expected.
Wasn't that intentional, as a mean to proper track WAL files still open for write? For example, in case of replication, it should go as far as any entry got already appended, no? Ping @Apache9 who worked on this before to give more thoughts.

This guy contacted me offline and I confirmed that this should be a problem.

What I can recall is that, when doing some bug fixes and improving the performance in AsyncFSWAL, I changed the way we calculate the length of the writer. Maybe I forget the assumption in HBASE-14004 when doing these changes and lead to the problem.

So @comnetwork , please add more comments to say why we need the getSyncedLength method in the WAL.Writer interface? So later people will not break it again.

Thanks.

Apache-HBase · 2020-06-25T14:11:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	7m 5s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 5s	master passed
+1 💚	checkstyle	1m 25s	master passed
+1 💚	spotbugs	2m 41s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 47s	the patch passed
-0 ⚠️	checkstyle	0m 11s	hbase-asyncfs: The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1)
-0 ⚠️	checkstyle	1m 11s	hbase-server: The patch generated 3 new + 38 unchanged - 0 fixed = 41 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 26s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 5s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 22s	The patch does not generate ASF License warnings.
		44m 54s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 99ad00f1b065 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1378776`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-25T14:15:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	6m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 45s	master passed
+1 💚	compile	1m 29s	master passed
+1 💚	shadedjars	6m 54s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 21s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 49s	hbase-server in master failed.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	5m 1s	the patch passed
+1 💚	compile	1m 36s	the patch passed
+1 💚	javac	1m 36s	the patch passed
+1 💚	shadedjars	6m 28s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 42s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 39s	hbase-asyncfs in the patch passed.
-1 ❌	unit	10m 22s	hbase-server in the patch failed.
		48m 37s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 66ca5e2db40c 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1378776`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/testReport/
Max. process+thread count	714 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-25T16:14:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 56s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 35s	master passed
+1 💚	compile	1m 17s	master passed
+1 💚	shadedjars	5m 37s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 53s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 25s	the patch passed
+1 💚	compile	1m 15s	the patch passed
+1 💚	javac	1m 15s	the patch passed
+1 💚	shadedjars	5m 35s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 49s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 30s	hbase-asyncfs in the patch passed.
+1 💚	unit	136m 58s	hbase-server in the patch passed.
		167m 45s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4ec9d41f5890 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1378776`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/testReport/
Max. process+thread count	4216 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

comnetwork · 2020-06-26T04:05:38Z

That is to say, AsyncFSWAL.getLogFileSizeIfBeingWritten could not reflect the file length which successfully synced to underlying HDFS, which is not as expected.
Wasn't that intentional, as a mean to proper track WAL files still open for write? For example, in case of replication, it should go as far as any entry got already appended, no? Ping @Apache9 who worked on this before to give more thoughts.
This guy contacted me offline and I confirmed that this should be a problem.
What I can recall is that, when doing some bug fixes and improving the performance in AsyncFSWAL, I changed the way we calculate the length of the writer. Maybe I forget the assumption in HBASE-14004 when doing these changes and lead to the problem.
So @comnetwork , please add more comments to say why we need the getSyncedLength method in the WAL.Writer interface? So later people will not break it again.
Thanks.

@Apache9 , I already added comments for WriteBase.getSyncedLength like following:

/**

NOTE: We add this method for {@link WALFileLengthProvider} used for replication, considering the
case if we use {@link AsyncFSWAL},we write to 3 DNs concurrently,according to the visibility
guarantee of HDFS, the data will be available immediately when arriving at DN since all the DNs
will be considered as the last one in pipeline. This means replication may read uncommitted data
and replicate it to the remote cluster and cause data inconsistency.
The method {@link WriterBase#getLength} may return length which just in hdfs client buffer and not
successfully synced to HDFS, so we use this method to return the length successfully synced to HDFS
and replication thread could only read writing WAL file limited by this length.
see also HBASE-14004 and this document for more details:
https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#
*/

Apache-HBase · 2020-06-26T04:41:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 9s	master passed
+1 💚	checkstyle	1m 32s	master passed
+1 💚	spotbugs	2m 47s	master passed
-0 ⚠️	patch	2m 26s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 48s	the patch passed
-0 ⚠️	checkstyle	0m 10s	hbase-asyncfs: The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1)
-0 ⚠️	checkstyle	1m 15s	hbase-server: The patch generated 8 new + 38 unchanged - 0 fixed = 46 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 36s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 24s	The patch does not generate ASF License warnings.
		41m 29s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 699d6a87ab5d 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `84e246f`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-26T06:38:13Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 32s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 58s	master passed
+1 💚	compile	1m 24s	master passed
+1 💚	shadedjars	5m 49s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 17s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 42s	hbase-server in master failed.
-0 ⚠️	patch	7m 11s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 7s	the patch passed
+1 💚	compile	1m 23s	the patch passed
+1 💚	javac	1m 23s	the patch passed
+1 💚	shadedjars	5m 45s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 17s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 39s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 25s	hbase-asyncfs in the patch passed.
+1 💚	unit	128m 28s	hbase-server in the patch passed.
		157m 57s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b55e15b83043 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `84e246f`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/testReport/
Max. process+thread count	3870 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-26T07:54:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 8s	master passed
+1 💚	compile	1m 22s	master passed
+1 💚	shadedjars	6m 20s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 51s	master passed
-0 ⚠️	patch	7m 31s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 58s	the patch passed
+1 💚	compile	1m 17s	the patch passed
+1 💚	javac	1m 17s	the patch passed
+1 💚	shadedjars	6m 9s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 36s	hbase-server generated 9 new + 28 unchanged - 0 fixed = 37 total (was 28)
		_ Other Tests _
+1 💚	unit	1m 41s	hbase-asyncfs in the patch passed.
+1 💚	unit	203m 15s	hbase-server in the patch passed.
		234m 7s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8389f6cd5833 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `84e246f`
Default Java	1.8.0_232
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/testReport/
Max. process+thread count	3555 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-26T08:44:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 4s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 25s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 13s	master passed
+1 💚	checkstyle	1m 31s	master passed
+1 💚	spotbugs	2m 51s	master passed
-0 ⚠️	patch	2m 28s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 57s	the patch passed
-0 ⚠️	checkstyle	0m 13s	hbase-asyncfs: The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1)
-0 ⚠️	checkstyle	1m 15s	hbase-server: The patch generated 8 new + 38 unchanged - 0 fixed = 46 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 55s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 41s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 27s	The patch does not generate ASF License warnings.
		42m 46s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 458640ec86b3 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c046120`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-26T11:48:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 43s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 30s	Maven dependency ordering for branch
+1 💚	mvninstall	5m 5s	master passed
+1 💚	compile	1m 39s	master passed
+1 💚	shadedjars	7m 6s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 18s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 44s	hbase-server in master failed.
-0 ⚠️	patch	8m 27s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 57s	the patch passed
+1 💚	compile	1m 57s	the patch passed
+1 💚	javac	1m 57s	the patch passed
+1 💚	shadedjars	7m 0s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 15s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 48s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 44s	hbase-asyncfs in the patch passed.
+1 💚	unit	190m 8s	hbase-server in the patch passed.
		226m 7s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 039abb9fd522 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c046120`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/testReport/
Max. process+thread count	3498 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-26T11:57:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 26s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 14s	master passed
+1 💚	compile	1m 22s	master passed
+1 💚	shadedjars	6m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 54s	master passed
-0 ⚠️	patch	7m 35s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 0s	the patch passed
+1 💚	compile	1m 20s	the patch passed
+1 💚	javac	1m 20s	the patch passed
+1 💚	shadedjars	6m 7s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 36s	hbase-server generated 9 new + 28 unchanged - 0 fixed = 37 total (was 28)
		_ Other Tests _
+1 💚	unit	1m 49s	hbase-asyncfs in the patch passed.
+1 💚	unit	203m 1s	hbase-server in the patch passed.
		234m 54s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9f9c61cfcf35 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c046120`
Default Java	1.8.0_232
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/testReport/
Max. process+thread count	3206 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2020-06-30T15:20:37Z


 protected FSDataOutputStream output;

+ private volatile long syncedLength = 0;


Use AtomicUtils.updateMax. It is a util class in hbase-common.

comnetwork · 2020-07-02T02:48:08Z

@Apache9 , Used AtomicUtils to replace synchronized, please have a review again, thanks.

Apache-HBase · 2020-07-02T03:04:13Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 31s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 52s	master passed
+1 💚	checkstyle	1m 25s	master passed
+1 💚	spotbugs	2m 39s	master passed
-0 ⚠️	patch	2m 20s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 42s	the patch passed
-0 ⚠️	checkstyle	0m 11s	hbase-asyncfs: The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1)
-0 ⚠️	checkstyle	1m 12s	hbase-server: The patch generated 8 new + 38 unchanged - 0 fixed = 46 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 18s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 58s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 22s	The patch does not generate ASF License warnings.
		38m 44s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux b021aa8a3afa 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `afe2eac`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/5/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-07-02T10:35:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 54s	master passed
+1 💚	compile	1m 31s	master passed
+1 💚	shadedjars	6m 25s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 18s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 41s	hbase-server in master failed.
-0 ⚠️	patch	7m 48s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 38s	the patch passed
+1 💚	compile	1m 36s	the patch passed
+1 💚	javac	1m 36s	the patch passed
+1 💚	shadedjars	6m 26s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 40s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 37s	hbase-asyncfs in the patch passed.
+1 💚	unit	191m 46s	hbase-server in the patch passed.
		225m 7s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux fde175799407 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `afe2eac`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/testReport/
Max. process+thread count	3011 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-07-02T10:42:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 54s	master passed
+1 💚	compile	1m 21s	master passed
+1 💚	shadedjars	6m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 50s	master passed
-0 ⚠️	patch	7m 27s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 4s	the patch passed
+1 💚	compile	1m 17s	the patch passed
+1 💚	javac	1m 17s	the patch passed
+1 💚	shadedjars	6m 9s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 40s	hbase-server generated 9 new + 28 unchanged - 0 fixed = 37 total (was 28)
		_ Other Tests _
+1 💚	unit	1m 43s	hbase-asyncfs in the patch passed.
+1 💚	unit	202m 10s	hbase-server in the patch passed.
		232m 54s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c5e680611bf6 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `afe2eac`
Default Java	1.8.0_232
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/testReport/
Max. process+thread count	3659 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/6/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache9

LGTM. But please fix the javadoc issues? After you fix the javadoc issues I will merge the PR.

Thanks.

Apache9 · 2020-07-02T13:12:54Z

 interface WriterBase extends Closeable {
 long getLength();
+ /**
+ * <pre>


Do not need to use pre here? These are just normal text.

ok, I would fix it.

Apache9 · 2020-07-03T13:38:12Z

Hi, any updates here? @comnetwork
Maybe we should include this in 2.3? It is a nasty bug... @ndimiduk

comnetwork · 2020-07-03T14:22:10Z

Removed the pre tag in the comment and fix the javadoc bug.

Apache-HBase · 2020-07-03T15:07:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 55s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 13s	master passed
+1 💚	checkstyle	1m 28s	master passed
+1 💚	spotbugs	2m 46s	master passed
-0 ⚠️	patch	2m 26s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 55s	the patch passed
-0 ⚠️	checkstyle	0m 13s	hbase-asyncfs: The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1)
-0 ⚠️	checkstyle	1m 15s	hbase-server: The patch generated 9 new + 38 unchanged - 0 fixed = 47 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 58s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 45s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 26s	The patch does not generate ASF License warnings.
		42m 19s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 96f93069c773 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2020-07-03T15:10:10Z

There are still checkstyle issues?

Apache-HBase · 2020-07-03T18:12:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	6m 49s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 29s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 35s	master passed
+1 💚	compile	1m 27s	master passed
+1 💚	shadedjars	6m 16s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 40s	hbase-server in master failed.
-0 ⚠️	patch	7m 30s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 34s	the patch passed
+1 💚	compile	1m 27s	the patch passed
+1 💚	javac	1m 27s	the patch passed
+1 💚	shadedjars	6m 17s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 40s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 35s	hbase-asyncfs in the patch passed.
+1 💚	unit	189m 55s	hbase-server in the patch passed.
		227m 19s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f23863ee3602 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/testReport/
Max. process+thread count	3600 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-07-03T18:21:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 10s	master passed
+1 💚	compile	1m 22s	master passed
+1 💚	shadedjars	6m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 52s	master passed
-0 ⚠️	patch	7m 26s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 56s	the patch passed
+1 💚	compile	1m 20s	the patch passed
+1 💚	javac	1m 20s	the patch passed
+1 💚	shadedjars	6m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 50s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 48s	hbase-asyncfs in the patch passed.
+1 💚	unit	205m 13s	hbase-server in the patch passed.
		236m 10s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b2e71be38d2d 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/testReport/
Max. process+thread count	3037 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/7/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

comnetwork · 2020-07-04T05:09:08Z

Fixed the checkstyle errors.

Apache-HBase · 2020-07-04T05:59:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 35s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 26s	master passed
+1 💚	checkstyle	1m 34s	master passed
+1 💚	spotbugs	3m 9s	master passed
-0 ⚠️	patch	2m 49s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 20s	the patch passed
+1 💚	checkstyle	1m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	13m 40s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 27s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		43m 32s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux b5df25e967f7 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-07-04T07:53:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 23s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 11s	master passed
+1 💚	compile	1m 24s	master passed
+1 💚	shadedjars	5m 43s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 19s	hbase-asyncfs in master failed.
-0 ⚠️	javadoc	0m 38s	hbase-server in master failed.
-0 ⚠️	patch	7m 4s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 58s	the patch passed
+1 💚	compile	1m 26s	the patch passed
+1 💚	javac	1m 26s	the patch passed
+1 💚	shadedjars	5m 45s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	hbase-asyncfs in the patch failed.
-0 ⚠️	javadoc	0m 38s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	1m 25s	hbase-asyncfs in the patch passed.
+1 💚	unit	128m 38s	hbase-server in the patch passed.
		157m 41s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ef12e9adeb43 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/testReport/
Max. process+thread count	4032 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-07-04T08:00:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 23s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 36s	master passed
+1 💚	compile	1m 16s	master passed
+1 💚	shadedjars	5m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 53s	master passed
-0 ⚠️	patch	6m 51s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 25s	the patch passed
+1 💚	compile	1m 13s	the patch passed
+1 💚	javac	1m 13s	the patch passed
+1 💚	shadedjars	5m 31s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 52s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 31s	hbase-asyncfs in the patch passed.
+1 💚	unit	137m 49s	hbase-server in the patch passed.
		165m 3s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1970
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b8cee7d31178 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e614b89`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/testReport/
Max. process+thread count	4405 (vs. ulimit of 12500)
modules	C: hbase-asyncfs hbase-server U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1970/8/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…he expected synced file length. (#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…he expected synced file length. (apache#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

ndimiduk · 2020-07-06T19:29:39Z

- future.complete(out.getPos());
+ long pos = out.getPos();
+ if(pos > this.syncedLength) {
+ this.syncedLength = pos;


This read-followedby-update also needs to be atomic, yes?

This one is just for test so not a big problem but aligning with other producation implementations is better. Can have an addendum.

Oh, reviewed the code again, actuall, the flush0 method can only be executed in a single thread so no need to use AtomicUtils.updateMax. The AtomicLong is in the ProtobufLogWriter, not the output stream. But the 'if(pos > this.syncedLength) {' is a bit confusing to developers, I prefer we just remove this check...

…return the expected synced file length. (#1970)" This reverts commit f834919.

…return the expected synced file length. (#1970)" This reverts commit bf587fa.

…he expected synced file length. (apache#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…return the expected synced file length. (apache#1970)" This reverts commit f834919.

…he expected synced file length. (apache#1970) Author: chenglei Reason: Bug Ref: CDPD-15964 Signed-off-by: Duo Zhang <zhangduo@apache.org> Change-Id: Ie87d5e6f1eb47c48f413a31a0e5507b0090f5fe1 (cherry picked from commit e064d08)

…he expected synced file length. (apache#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit bf587fa) Change-Id: Ie87d5e6f1eb47c48f413a31a0e5507b0090f5fe1

…return the expected synced file length. (apache#1970)" This reverts commit bf587fa. (cherry picked from commit 160c229) Change-Id: Ifffde0910e6a219956697f417b76da31816a7852

HBASE-24625
60abb39

wchevreuil reviewed Jun 25, 2020
View reviewed changes

Apache9 requested changes Jun 25, 2020
View reviewed changes

Apache9 changed the title ~~HBASE-24625~~HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.Jun 25, 2020

comnetwork added 2 commits June 26, 2020 11:41

HBASE-24625
640aa1b

Merge branch 'master' ofhttps://github.com/comnetwork/hbase
de20b66

comnetwork requested a review from Apache9 June 26, 2020 04:32

Merge remote-tracking branch 'upstream/master'
b9f1572

Apache9 requested changes Jun 30, 2020
View reviewed changes

comnetwork added 3 commits July 2, 2020 10:16

Merge remote-tracking branch 'upstream/master'
3c68fa1

replace synchronized with AtomicUtils
969898c

replace synchronized with AtomicUtils(addnum)
a31b866

comnetwork requested a review from Apache9 July 2, 2020 02:46

Apache9 approved these changes Jul 2, 2020
View reviewed changes

comnetwork added 2 commits July 3, 2020 22:16

Merge remote-tracking branch 'upstream/master'
715d7d5

remove pre
f78c54a

fix checkstyle error
f33ed00

Apache9 merged commit a7a0e1a into apache:masterJul 4, 2020

asfgit pushed a commit that referenced this pull request Jul 4, 2020
HBASE-24625AsyncFSWAL.getLogFileSizeIfBeingWritten does not return t…
f834919
…he expected synced file length. (#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

asfgit pushed a commit that referenced this pull request Jul 4, 2020
HBASE-24625AsyncFSWAL.getLogFileSizeIfBeingWritten does not return t…
bf587fa
…he expected synced file length. (#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jul 6, 2020
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return t…
bfb169f
…he expected synced file length. (apache#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

ndimiduk reviewed Jul 6, 2020
View reviewed changes

asfgit pushed a commit that referenced this pull request Jul 7, 2020
Revert "HBASE-24625AsyncFSWAL.getLogFileSizeIfBeingWritten does not …
28602d6
…return the expected synced file length. (#1970)" This reverts commit f834919.

asfgit pushed a commit that referenced this pull request Jul 7, 2020
Revert "HBASE-24625AsyncFSWAL.getLogFileSizeIfBeingWritten does not …
160c229
…return the expected synced file length. (#1970)" This reverts commit bf587fa.

comnetwork mentioned this pull request Jul 7, 2020
Backport "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)" to branch-2 #2034
Merged

comnetwork mentioned this pull request Jul 15, 2020
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length(addendum) #2055
Merged

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return t…
e8269d6
…he expected synced file length. (apache#1970) Signed-off-by: Duo Zhang <zhangduo@apache.org>

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
Revert "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not …
4c12ce9
…return the expected synced file length. (apache#1970)" This reverts commit f834919.


		protected FSDataOutputStream output;

		private volatile long syncedLength = 0;

Uh oh!

Conversation

comnetwork commented Jun 25, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

wchevreuil commented Jun 25, 2020

Uh oh!

wchevreuil left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Apache9 commented Jun 25, 2020

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

Apache-HBase commented Jun 25, 2020

Uh oh!

comnetwork commented Jun 26, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Apache-HBase commented Jun 26, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

comnetwork commented Jul 2, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Jul 2, 2020

Uh oh!

Apache-HBase commented Jul 2, 2020

Uh oh!

Apache-HBase commented Jul 2, 2020

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Apache9 commented Jul 3, 2020

Uh oh!

comnetwork commented Jul 3, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Jul 3, 2020

Uh oh!

Apache9 commented Jul 3, 2020

Uh oh!

Apache-HBase commented Jul 3, 2020

Uh oh!

Apache-HBase commented Jul 3, 2020

Uh oh!

comnetwork commented Jun 25, 2020•
edited
Loading

comnetwork commented Jun 26, 2020•
edited
Loading

comnetwork commented Jul 2, 2020•
edited
Loading

comnetwork commented Jul 3, 2020•
edited
Loading