[SPARK-7200] Check that memory is not leaked in TaskMemoryManagerSuite's after() by jsoltren · Pull Request #17402 · apache/spark

jsoltren · 2017-03-23T22:16:46Z

Cleans up all memory used by tests explicitly in after(). This new
function has the JUnit @after tag and thus is guaranteed to run, even
if an @test method throws an exception.

Testing:
Ran tests locally.
Hacked an existing test and made sure that a failure was reported in
the original test, and not in after().
Verified test logging in
test-reports/org.apache.spark.memory.TaskMemoryManagerSuite.xml.

What changes were proposed in this pull request?

Updates TaskMemoryManagerSuite based on Josh Rosen's comments in SPARK-7200. It would be good to get some feedback as to whether or not this is the right direction.

How was this patch tested?

See "Testing" above.

@after

…ryManagerSuite Cleans up all memory used by tests explicitly in after(). This new function has the JUnit @after tag and thus is guaranteed to run, even if an @test method throws an exception. Testing: Ran tests locally. Hacked an existing test and made sure that a failure was reported in the original test, and not in after(). Verified test logging in test-reports/org.apache.spark.memory.TaskMemoryManagerSuite.xml.

vanzin · 2017-03-23T23:43:37Z

ok to test

SparkQA · 2017-03-24T02:26:34Z

Test build #75129 has finished for PR 17402 at commit c44fc87.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

vanzin

Please update the title. You're not adding a cleanup, you're adding a check that no memory was leaked after tests ran.

vanzin · 2017-03-28T17:55:42Z


 public class TaskMemoryManagerSuite {

+ static TaskMemoryManager manager;


Does not need to be static. Also should be private.

vanzin · 2017-03-28T17:56:23Z

+
+ @After
+ public void after() {
+ Assert.assertEquals(0, manager.getMemoryConsumptionForThisTask());


Add Assert.assertNotNull(manager).
Also, probably minor, but if a test fails, these checks will probably fail in weird ways and might mask the original error... I'd try to insert a failure in a test and seeing what happens.

@squito had the same concern. I purposefully leaked some memory at the end of leakedPageMemoryIsDetected and got this very helpful trace:
[info] Test org.apache.spark.memory.TaskMemoryManagerSuite.leakedPageMemoryIsDetected started
[error] Test org.apache.spark.memory.TaskMemoryManagerSuite.leakedPageMemoryIsDetected failed: java.lang.AssertionError: expected:<0> but was:<4096>, took 0.007 sec
[error] at org.apache.spark.memory.TaskMemoryManagerSuite.after(TaskMemoryManagerSuite.java:34)
[error] ...

This sounds like this assert triggering, not the actual test failing?
I'm talking about is leaking memory and inserting a fail("blah") in a test and seeing what's reported.

vanzin · 2017-03-28T17:58:53Z

+final MemoryBlock block = manager.allocatePage(4096, c); // leak memory
 Assert.assertEquals(4096, manager.getMemoryConsumptionForThisTask());
 Assert.assertEquals(4096, manager.cleanUpAllAllocatedMemory());
+ manager.freePage(block, c);


Is this really needed? Why isn't the call above freeing this memory?

It should. We can check for that instead with:
Assert.assertEquals(0, manager.cleanUpAllAllocatedMemory());

HyukjinKwon · 2017-05-11T14:52:01Z

Hi @jsoltren, is this still active?

jsoltren · 2017-05-11T14:55:28Z

I'm still waiting on feedback from @JoshRosen on my earlier question. I can update this patch in response to @vanzin's review feedback if it would help to get this checked in.

IMO this is a pretty trivial fix that we should get checked in regardless.

HyukjinKwon · 2017-05-11T14:57:57Z

I see. Thank you for your input.

jsoltren · 2017-05-16T23:46:31Z

At this point I'd like to get this checked in whether or not @JoshRosen chimes in here. @HyukjinKwon @vanzin I've updated this based on the previous review feedback received. Thanks!

JoshRosen · 2017-05-17T00:34:46Z

I think that the scope / intent of the original SPARK-7200 JIRA was to include this type of logic as a generic assertion in multiple suites so that we gain additional implicit assertions in suites which would otherwise maybe not have them, whereas this patch looks mostly like cleanup/refactoring of a single suite's existing assertions.

I don't have any objections to this patch but I don't see it as a big win either. I don't think that this PR's specific changes will actually help to prevent any bugs because we don't appear to be adding assertions which weren't already present in the existing code.

SparkQA · 2017-05-17T02:33:19Z

Test build #76992 has finished for PR 17402 at commit e6ca027.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

vanzin · 2017-06-01T23:35:33Z

@jsoltren do you plan to follow up on Josh's comments? Seems like the scope of the bug was a little larger than you thought.

jsoltren · 2017-06-02T01:47:12Z

Yes, I'll post an updated change once I wrap up a couple of other things...

Seems like the scope of the bug was a little larger than you thought.

@vanzin The scope of the bug (specifically the text "after each suite") was vague so this was a purposefully small proof-of-concept change. I tried to be clear about this in JIRA comments and in the PR description.

That being said: yes, the direction I'll go is reviewing more existing tests and seeing where it would make sense to put a check for leaked memory. Ideally I'd put this in an after() or afterAll() block but I need to do some more digging to figure out exactly where.

jiangxb1987 · 2017-10-15T15:14:05Z

ping @jsoltren

vanzin · 2017-10-16T01:18:20Z

Pretty sure we can close this out.

Closesapache#11494Closesapache#14158Closesapache#16803Closesapache#16864Closesapache#17455Closesapache#17936Closesapache#19377 Added: Closesapache#19380Closesapache#18642Closesapache#18377Closesapache#19632 Added: Closesapache#14471Closesapache#17402Closesapache#17953Closesapache#18607 Also cc srowen vanzin HyukjinKwon gatorsmile cloud-fan to see if you have other PRs to close. Author: Xingbo Jiang <xingbo.jiang@databricks.com> Closesapache#19669 from jiangxb1987/stale-prs.

vanzin reviewed Mar 28, 2017
View reviewed changes

jsoltren changed the title ~~[SPARK-7200] Add cleanup for memory leak tests to after() in TaskMemo…~~[SPARK-7200] Check that memory is not leaked in TaskMemoryManagerSuite's after()May 16, 2017

jsoltren added 3 commits May 16, 2017 16:28

changes in response to code review from vanzin on 2017-03-28
8677a99

Merge remote-tracking branch 'upstream/master' into jose-tungsten
b3ffb7d

minor cleanup to get rid of unused temp variable
e6ca027

HyukjinKwon mentioned this pull request Nov 6, 2017
[BUILD] Close stale PRs #19669
Closed

asfgit closed this in ed1478cNov 7, 2017


		public class TaskMemoryManagerSuite {

		static TaskMemoryManager manager;

Uh oh!

Conversation

jsoltren commented Mar 23, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

vanzin commented Mar 23, 2017

Uh oh!

SparkQA commented Mar 24, 2017

Uh oh!

vanzin left a comment

Choose a reason for hiding this comment

Uh oh!

vanzinMar 28, 2017

Choose a reason for hiding this comment

Uh oh!

vanzinMar 28, 2017

Choose a reason for hiding this comment

Uh oh!

jsoltrenMay 16, 2017

Choose a reason for hiding this comment

Uh oh!

vanzinMay 17, 2017

Choose a reason for hiding this comment

Uh oh!

vanzinMar 28, 2017

Choose a reason for hiding this comment

Uh oh!

jsoltrenMay 16, 2017

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented May 11, 2017

Uh oh!

jsoltren commented May 11, 2017

Uh oh!

HyukjinKwon commented May 11, 2017

Uh oh!

jsoltren commented May 16, 2017

Uh oh!

JoshRosen commented May 17, 2017

Uh oh!

SparkQA commented May 17, 2017

Uh oh!

vanzin commented Jun 1, 2017

Uh oh!

jsoltren commented Jun 2, 2017

Uh oh!

jiangxb1987 commented Oct 15, 2017

Uh oh!

vanzin commented Oct 16, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jsoltren commented Mar 23, 2017•
edited
Loading