[SPARK-16409] [SQL] regexp_extract with optional groups causes NPE

[srowen]

What changes were proposed in this pull request?

regexp_extract actually returns null when it shouldn't when a regex matches but the requested optional group did not. This makes it return an empty string, as apparently designed.

How was this patch tested?

Additional unit test

[SparkQA]

Test build #63258 has finished for PR 14504 at commit 75d62ae.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #63310 has finished for PR 14504 at commit 545c8de.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #63313 has finished for PR 14504 at commit b835bd3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged to master/2.0/1.6

[ericl]

I think this broke the build: [error] /home/jenkins/workspace/spark-branch-1.6-compile-maven-with-yarn-2.4/sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala:82: value toDF is not a member of Seq[String] [error] val df = Seq("aaaac").toDF("s") [error] ^ [warn] two warnings found [error] one error found

[srowen]

Yes, for 1.6: #14526

[rxin]

@srowen did anybody review this?

[srowen]

I proceeded for lack of comments, and because this reproduces the Max's test case in the JIRA.