PHOENIX-6335 Jenkins tests getting OS level OOM erros by stoty · Pull Request #1106 · apache/phoenix

stoty · 2021-01-22T10:25:14Z

No description provided.

stoty · 2021-01-22T10:40:59Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/1/console in case of problems.

stoty · 2021-01-22T13:17:19Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/2/console in case of problems.

stoty · 2021-01-22T13:38:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 1s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	4m 24s	Maven dependency ordering for branch
+0	hbaserecompile	11m 27s	HBase recompiled.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
+0	hbaserecompile	7m 28s	HBase recompiled.
+1 💚	shellcheck	0m 2s	The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	shelldocs	0m 2s	There were no new shelldocs issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
		_ Other Tests _
+0 🆗	asflicense	0m 0s	ASF License check generated no output?
		21m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1106
Optional Tests	dupname asflicense shellcheck shelldocs
uname	Linux f7391c97f670 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `11b37e6`
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/2/console
versions	git=2.7.4 maven=3.3.9 shellcheck=0.7.0
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

stoty · 2021-01-22T13:44:26Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/3/console in case of problems.

virajjasani · 2021-01-22T14:26:33Z

+ DOCKER_MEMORY=20g
+
+ #Allow some swaping to avoid OOM errors
+ DOCKER_EXTRAARGS+=( "--memory-swap=24g" )


Is this a proven way? (Just asking in case any Hadoop project is following this and it's working fine for them)

I was just looking up https://docs.docker.com/config/containers/resource_constraints/ and found this:
--memory-swap is a modifier flag that only has meaning if --memory is also set.
What we specify with DOCKERMEMLIMIT is treated as --memory by Yetus?

Is this a proven way? (Just asking in case any Hadoop project is following this and it's working fine for them)
Not that I know of.
I was just looking at the docker memory management options, and this seemed like a solution without overcommitting the host memory.

What we specify with DOCKERMEMLIMIT is treated as --memory by Yetus?
Yes, DOCKER_MEMORY is passed as --memory= to docker by yetus.
(The DOCKERMEMLIMIT variable name was wrong, or at least obsolete)

stoty · 2021-01-22T19:19:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 16s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	16m 55s	master passed
+0	hbaserecompile	23m 53s	HBase recompiled.
+1 💚	compile	1m 11s	master passed
+1 💚	checkstyle	0m 33s	master passed
+1 💚	javadoc	0m 55s	master passed
+0 🆗	spotbugs	3m 19s	phoenix-core in master has 955 extant spotbugs warnings.
		_ Patch Compile Tests _
+1 💚	mvninstall	10m 54s	the patch passed
+0	hbaserecompile	23m 41s	HBase recompiled.
+1 💚	compile	1m 39s	the patch passed
+1 💚	javac	1m 39s	the patch passed
+1 💚	checkstyle	0m 48s	the patch passed
+1 💚	shellcheck	0m 2s	The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	shelldocs	0m 4s	There were no new shelldocs issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	1m 25s	the patch passed
+1 💚	spotbugs	5m 38s	the patch passed
		_ Other Tests _
-1 ❌	unit	269m 42s	phoenix-core in the patch failed.
+1 💚	asflicense	1m 3s	The patch does not generate ASF License warnings.
		335m 25s

Reason	Tests
Failed junit tests	phoenix.end2end.index.LocalMutableNonTxIndexIT
	phoenix.end2end.index.MutableIndexIT

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1106
Optional Tests	dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname	Linux 5f3d762a9612 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `11b37e6`
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/3/testReport/
Max. process+thread count	10467 (vs. ulimit of 30000)
modules	C: phoenix-core U: phoenix-core
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/3/console
versions	git=2.7.4 maven=3.3.9 shellcheck=0.7.0 spotbugs=4.1.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…ocal indexed tables

stoty · 2021-08-05T12:00:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	7m 26s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	24m 7s	master passed
+0	hbaserecompile	31m 19s	HBase recompiled.
+1 💚	compile	1m 9s	master passed
+1 💚	checkstyle	1m 15s	master passed
+1 💚	javadoc	0m 53s	master passed
+0 🆗	spotbugs	3m 26s	phoenix-core in master has 973 extant spotbugs warnings.
		_ Patch Compile Tests _
+1 💚	mvninstall	17m 0s	the patch passed
+0	hbaserecompile	27m 44s	HBase recompiled.
+1 💚	compile	1m 5s	the patch passed
+1 💚	javac	1m 5s	the patch passed
-1 ❌	checkstyle	1m 12s	phoenix-core: The patch generated 44 new + 556 unchanged - 36 fixed = 600 total (was 592)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 49s	the patch passed
-1 ❌	spotbugs	3m 39s	phoenix-core generated 1 new + 973 unchanged - 0 fixed = 974 total (was 973)
		_ Other Tests _
-1 ❌	unit	231m 22s	phoenix-core in the patch failed.
+1 💚	asflicense	0m 35s	The patch does not generate ASF License warnings.
		311m 29s

Reason	Tests
FindBugs	module:phoenix-core
	Load of known null value in org.apache.phoenix.compile.FromCompiler$ProjectedTableColumnResolver.resolveColumn(String, String, String) At FromCompiler.java:in org.apache.phoenix.compile.FromCompiler$ProjectedTableColumnResolver.resolveColumn(String, String, String) At FromCompiler.java:[line 1202]
Failed junit tests	phoenix.tx.ParameterizedTransactionIT
	phoenix.end2end.index.ViewIndexIT
	phoenix.end2end.FlappingLocalIndexIT
	phoenix.end2end.index.LocalIndexIT

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1106
Optional Tests	dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname	Linux 6bed6d850187 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `025a4af`
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
spotbugs	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/testReport/
Max. process+thread count	11990 (vs. ulimit of 30000)
modules	C: phoenix-core U: phoenix-core
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1106/4/console
versions	git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

stoty · 2023-08-22T14:13:57Z

Most of this doesn't even look relevant to memory usage.
I seem to have mixed up some branches.

stoty force-pushed the PHOENIX-6335 branch from 101bd86 to b8c7b25Compare January 22, 2021 13:15

stoty force-pushed the PHOENIX-6335 branch from b8c7b25 to ce07f58Compare January 22, 2021 13:43

virajjasani reviewed Jan 22, 2021
View reviewed changes

PHOENIX-6365 Bogus AmbiguousTableException in query with aliases on l…
c2874b5
…ocal indexed tables

stoty force-pushed the PHOENIX-6335 branch from ce07f58 to c2874b5Compare August 5, 2021 06:48

stoty closed this Aug 22, 2023

Navigation Menu

Uh oh!

PHOENIX-6335 Jenkins tests getting OS level OOM erros - #1106

PHOENIX-6335 Jenkins tests getting OS level OOM erros#1106
stoty wants to merge 1 commit into
apache:masterfrom
stoty:PHOENIX-6335

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

virajjasaniJan 22, 2021•
edited
Loading

Uh oh!

virajjasaniJan 22, 2021

Uh oh!

stotyJan 22, 2021

Uh oh!

stotyJan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Aug 5, 2021

Uh oh!

stoty commented Aug 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

virajjasaniJan 22, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

virajjasaniJan 22, 2021

Choose a reason for hiding this comment

Uh oh!

stotyJan 22, 2021

Choose a reason for hiding this comment

Uh oh!

stotyJan 22, 2021

Choose a reason for hiding this comment

Uh oh!

stoty commented Jan 22, 2021

Uh oh!

stoty commented Aug 5, 2021

Uh oh!

stoty commented Aug 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

virajjasaniJan 22, 2021•
edited
Loading