Skip to content

[SPARK-32160][CORE][PYSPARK][FOLLOWUP] Change the config name to switch allow/disallow SparkContext in executors. - #29340

Closed
ueshin wants to merge 2 commits into
apache:masterfrom
ueshin:issues/SPARK-32160/change_config_name
Closed

[SPARK-32160][CORE][PYSPARK][FOLLOWUP] Change the config name to switch allow/disallow SparkContext in executors.#29340
ueshin wants to merge 2 commits into
apache:masterfrom
ueshin:issues/SPARK-32160/change_config_name

Conversation

@ueshin

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This is a follow-up of #29278.
This PR changes the config name to switch allow/disallow SparkContext in executors as per the comment #29278 (review).

Why are the changes needed?

The config name spark.executor.allowSparkContext is more reasonable.

Does this PR introduce any user-facing change?

Yes, the config name is changed.

How was this patch tested?

Updated tests.

@dongjoon-hyundongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @ueshin !

@SparkQA

Copy link
Copy Markdown

Test build #127005 has finished for PR 29340 at commit 2602fe8.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

Copy link
Copy Markdown
Member

+1 I think this makes more sense.

@SparkQA

Copy link
Copy Markdown

Test build #127013 has finished for PR 29340 at commit 7e0f7fc.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

Copy link
Copy Markdown
Contributor

retest this please

@HyukjinKwon

Copy link
Copy Markdown
Member

I am merging this to get this in Spark 3.0.1. The test failure in Kafka will be fixed at #29343.

@HyukjinKwon

Copy link
Copy Markdown
Member

Merged to master.

@dongjoon-hyun

Copy link
Copy Markdown
Member

Please note that there is a PR for the 170 Cache related failures.

@SparkQA

Copy link
Copy Markdown

Test build #127023 has finished for PR 29340 at commit 7e0f7fc.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #127021 has finished for PR 29340 at commit 7e0f7fc.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@ueshin
ueshin deleted the issues/SPARK-32160/change_config_name branch August 3, 2026 22:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ueshin@SparkQA@HyukjinKwon@cloud-fan@dongjoon-hyun