Skip to content

[SPARK-1087] Move python traceback utilities into new traceback_utils.py file. - #2385

Closed
staple wants to merge 2 commits into
apache:masterfrom
staple:SPARK-1087
Closed

[SPARK-1087] Move python traceback utilities into new traceback_utils.py file.#2385
staple wants to merge 2 commits into
apache:masterfrom
staple:SPARK-1087

Conversation

@staple

Copy link
Copy Markdown
Contributor

Also made some cosmetic cleanups.

@SparkQA

Copy link
Copy Markdown

Can one of the admins verify this patch?

@JoshRosen

Copy link
Copy Markdown
Contributor

Jenkins, this is ok to test.

@SparkQA

Copy link
Copy Markdown

QA tests have started for PR 2385 at commit 10ba6e1.

  • This patch merges cleanly.

@jyotiska

Copy link
Copy Markdown
Contributor

LGTM.

@SparkQA

Copy link
Copy Markdown

QA tests have finished for PR 2385 at commit 10ba6e1.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class JavaStackTrace(object):

@staple

Copy link
Copy Markdown
ContributorAuthor

Hi, the above failure in NetworkReceiverSuite.scala seems like it may be unrelated to this patch. That test also passed when I ran locally.

@SparkQA

Copy link
Copy Markdown

QA tests have started for PR 2385 at commit 10ba6e1.

  • This patch merges cleanly.

@SparkQA

Copy link
Copy Markdown

QA tests have finished for PR 2385 at commit 10ba6e1.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class JavaStackTrace(object):

Comment threadpython/pyspark/context.py Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better to only call extract_concise_traceback() once, such as:

self._callsite = extract_concise_traceback()
if self._callsite is None:
xxxx

@davies

Copy link
Copy Markdown
Contributor

LGTM, just one minor comment, it's not must to have.

Comment threadpython/pyspark/traceback_utils.py Outdated

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I also need to put JavaStackTrace here instead of SparkContext.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are just internal interfaces, so it's fine to not have all here. If having, it should be JavaStackTrace

@staple

Copy link
Copy Markdown
ContributorAuthor

Hi - I addressed the review comments and made some additional cosmetic changes.

@SparkQA

Copy link
Copy Markdown

QA tests have started for PR 2385 at commit 7b3bb13.

  • This patch merges cleanly.

@SparkQA

Copy link
Copy Markdown

QA tests have finished for PR 2385 at commit 7b3bb13.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class SCCallSiteSync(object):

@JoshRosen

Copy link
Copy Markdown
Contributor

This looks good to me, so I'm going to merge it. Thanks!

@staple

Copy link
Copy Markdown
ContributorAuthor

Great! Thanks to all the reviewers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@staple@SparkQA@JoshRosen@jyotiska@davies