[SPARK-25144][SQL][TEST] Free aggregate map when task ends

[dongjoon-hyun]

What changes were proposed in this pull request?

SPARK-25144 reports memory leaks on Apache Spark 2.0.2 ~ 2.3.2-RC5. The bug is already fixed via #21738 as a part of SPARK-21743. This PR only adds a test case to prevent any future regression.

scala>caseclassFoo(bar: Option[String])
scala>valds=List(Foo(Some("bar"))).toDS
scala>valresult= ds.flatMap(_.bar).distinct
scala> result.rdd.isEmpty
18/08/1923:01:54WARNExecutor:Managed memory leak detected; size =8650752 bytes, TID=125
res0:Boolean=false

How was this patch tested?

Pass the Jenkins with a new added test case.

[dongjoon-hyun]

cc @HyukjinKwon

[dongjoon-hyun]

Thank you for review and approval, @HyukjinKwon .

[SparkQA]

Test build #94962 has finished for PR 22155 at commit 27497f5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• case class Foo(bar: Option[String])

[HyukjinKwon]

Merged to master.

[dongjoon-hyun]

Thank you!