Skip to content

[SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example - #25956

Closed
MaxGekk wants to merge 1 commit into
apache:masterfrom
MaxGekk:fix-expr-examples-checks
Closed

[SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example#25956
MaxGekk wants to merge 1 commit into
apache:masterfrom
MaxGekk:fix-expr-examples-checks

Conversation

@MaxGekk

@MaxGekkMaxGekk commented Sep 28, 2019

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

In the PR, I propose to clone Spark session per-each expression example. Examples can modify SQL settings, and can influence on each other if they run in the same Spark session in parallel.

Why are the changes needed?

This should fix test failures like this checking of the Like example:

org.apache.spark.sql.AnalysisException: the pattern '\%SystemDrive\%\Users%' is invalid, the escape character is not allowed to precede 'U';
at org.apache.spark.sql.catalyst.util.StringUtils$.fail$1(StringUtils.scala:48)
at org.apache.spark.sql.catalyst.util.StringUtils$.escapeLikeRegex(StringUtils.scala:57)
at org.apache.spark.sql.catalyst.expressions.Like.escape(regexpExpressions.scala:108)

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running check outputs of expression examples in org.apache.spark.sql.SQLQuerySuite

@MaxGekkMaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP] Clone SparkSession per each function example[WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function exampleSep 28, 2019
@SparkQA

Copy link
Copy Markdown

Test build #111521 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #111522 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #111523 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

jenkins, retest this, please

@MaxGekkMaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example[SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function exampleSep 28, 2019
@SparkQA

Copy link
Copy Markdown

Test build #111526 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

jenkins, retest this, please

@SparkQA

Copy link
Copy Markdown

Test build #111528 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

jenkins, retest this, please

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

@HyukjinKwon@wangyum@dongjoon-hyun Take a look at this PR, please.

@SparkQA

Copy link
Copy Markdown

Test build #111530 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #111533 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

Copy link
Copy Markdown
Member

Merged to master.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@MaxGekk@SparkQA@HyukjinKwon@srowen