[SPARK-13747][SQL]Fix concurrent executions in ForkJoinPool for SQL by zsxwing · Pull Request #15520 · apache/spark

zsxwing · 2016-10-17T21:02:13Z

What changes were proposed in this pull request?

Calling Await.result will allow other tasks to be run on the same thread when using ForkJoinPool. However, SQL uses a ThreadLocal execution id to trace Spark jobs launched by a query, which doesn't work perfectly in ForkJoinPool.

This PR just uses Awaitable.result instead to prevent ForkJoinPool from running other tasks in the current waiting thread.

How was this patch tested?

Jenkins

jodersky · 2016-10-17T22:32:51Z

I'm not sure I completely understand why you have to use Awaitable instead of Await. Is it to avoid the blocking() call in Await?

zsxwing · 2016-10-17T22:34:41Z

Is it to avoid the blocking() call in Await?

Yep. Without blocking(), other tasks cannot run in the current thread during waiting for result, then ThreadLocal will not be leaked to other tasks.

jodersky · 2016-10-17T22:50:12Z

That makes sense, thanks for explaining. Would installing a custom BlockingContext on threads that run tasks also work? I'm not very familiar with the internals of the Awaitable api so I'm just throwing this out there in the hopes you know more :)

SparkQA · 2016-10-17T22:54:47Z

Test build #67088 has finished for PR 15520 at commit 54078ce.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

zsxwing · 2016-10-17T23:03:20Z

Would installing a custom BlockingContext on threads that run tasks also work?

Yes, I think so. It just needs more lines :)

zsxwing · 2016-10-17T23:04:24Z

retest this please

jodersky · 2016-10-17T23:12:07Z

Yes, I think so. It just needs more lines :)

I think the solution you propose is sound. Do you think that a quick mention about avoiding the BlockingContext in a comment would help in case someone sees this code in the future and is puzzled as to what it does?

SparkQA · 2016-10-18T01:22:46Z

Test build #67094 has finished for PR 15520 at commit 54078ce.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

zsxwing · 2016-10-18T05:25:58Z

cc @andrewor14

SparkQA · 2016-10-18T07:40:49Z

Test build #67112 has finished for PR 15520 at commit 6aa9e2f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2016-10-22T00:06:36Z

 // scalastyle:off awaitresult
 Await.result(...)
 // scalastyle:on awaitresult
+ If your codes use ThreadLocal and run in a user's thread, use ThreadUtils.awaitResultInForkJoinSafely instead.


What does the user's thread mean at here?

I meant threads created by the user. Fixed the description.

SparkQA · 2016-10-24T19:53:19Z

Test build #67461 has finished for PR 15520 at commit e9753ce.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2016-10-26T17:35:01Z

lgtm

zsxwing · 2016-10-26T17:36:23Z

Thanks! Merging to master and 2.0.

zsxwing · 2016-10-26T17:43:56Z

There are conflicts with 2.0. #15646 is the backport for 2.0.

…(branch 2.0) ## What changes were proposed in this pull request? Backport #15520 to 2.0. ## How was this patch tested? Jenkins Author: Shixiong Zhu <shixiong@databricks.com> Closes#15646 from zsxwing/SPARK-13747-2.0.

## What changes were proposed in this pull request? Calling `Await.result` will allow other tasks to be run on the same thread when using ForkJoinPool. However, SQL uses a `ThreadLocal` execution id to trace Spark jobs launched by a query, which doesn't work perfectly in ForkJoinPool. This PR just uses `Awaitable.result` instead to prevent ForkJoinPool from running other tasks in the current waiting thread. ## How was this patch tested? Jenkins Author: Shixiong Zhu <shixiong@databricks.com> Closesapache#15520 from zsxwing/SPARK-13747.

Fix concurrent executions in ForkJoinPool for SQL
54078ce

Update comment
6aa9e2f

yhuai reviewed Oct 22, 2016
View reviewed changes

Update
e9753ce

asfgit closed this in 7ac70e7Oct 26, 2016

zsxwing mentioned this pull request Oct 26, 2016
[SPARK-13747][SQL] Fix concurrent executions in ForkJoinPool for SQL (branch 2.0) #15646
Closed

zsxwing deleted the SPARK-13747 branch October 26, 2016 17:43

zsxwing mentioned this pull request Dec 9, 2016
[SPARK-13747][Core]Fix potential ThreadLocal leaks in RPC when using ForkJoinPool #16230
Closed

Uh oh!

Conversation

zsxwing commented Oct 17, 2016

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

jodersky commented Oct 17, 2016

Uh oh!

zsxwing commented Oct 17, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jodersky commented Oct 17, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Oct 17, 2016

Uh oh!

zsxwing commented Oct 17, 2016

Uh oh!

zsxwing commented Oct 17, 2016

Uh oh!

jodersky commented Oct 17, 2016

Uh oh!

SparkQA commented Oct 18, 2016

Uh oh!

zsxwing commented Oct 18, 2016

Uh oh!

SparkQA commented Oct 18, 2016

Uh oh!

yhuaiOct 22, 2016

Choose a reason for hiding this comment

Uh oh!

zsxwingOct 24, 2016

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 24, 2016

Uh oh!

yhuai commented Oct 26, 2016

Uh oh!

zsxwing commented Oct 26, 2016

Uh oh!

zsxwing commented Oct 26, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zsxwing commented Oct 17, 2016•
edited
Loading

jodersky commented Oct 17, 2016•
edited
Loading