Skip to content

[SPARK-19094][WIP][PySpark] Plumb through logging for IJupyter notebooks - #17298

Closed
holdenk wants to merge 29 commits into
apache:masterfrom
holdenk:SPARK-19094-plumb-through-logging-for-ijupyter-notebooks
Closed

[SPARK-19094][WIP][PySpark] Plumb through logging for IJupyter notebooks#17298
holdenk wants to merge 29 commits into
apache:masterfrom
holdenk:SPARK-19094-plumb-through-logging-for-ijupyter-notebooks

Conversation

@holdenk

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Plumb through logging from the JVM into Jupyter notebooks to make debugging easier. This is a WIP PR

How was this patch tested?

Modified standalone_tests to allow testing with Python launch first in addition to the JVM launch first default.

@SparkQA

Copy link
Copy Markdown

Test build #74568 has finished for PR 17298 at commit 15d999b.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk

Copy link
Copy Markdown
ContributorAuthor

jenkins retest this please.

@SparkQA

Copy link
Copy Markdown

Test build #75869 has finished for PR 17298 at commit 15d999b.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #75881 has finished for PR 17298 at commit 955f163.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #75882 has finished for PR 17298 at commit 2bfaa16.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76530 has finished for PR 17298 at commit 6c22a89.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76711 has finished for PR 17298 at commit a2caf97.

  • This patch fails Python style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76865 has finished for PR 17298 at commit 1166b9e.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76870 has finished for PR 17298 at commit 7fcc839.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76897 has finished for PR 17298 at commit 08632fd.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

…uring standalone tests after reseting the stderr pipe so the user can see it
@SparkQA

Copy link
Copy Markdown

Test build #76903 has finished for PR 17298 at commit 89cf739.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • case class ParseToTimestamp(left: Expression, format: Option[Expression], child: Expression)
  • class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging
  • class CatalystSqlParser(conf: SQLConf) extends AbstractSqlParser
  • case class ColumnStatsMap(originalMap: AttributeMap[ColumnStat])
  • trait DataSourceScanExec extends LeafExecNode with CodegenSupport with PredicateHelper
  • class SparkSqlAstBuilder(conf: SQLConf) extends AstBuilder(conf)
  • s\"($

@SparkQA

Copy link
Copy Markdown

Test build #77118 has finished for PR 17298 at commit bb36f50.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #77120 has finished for PR 17298 at commit 24ff579.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #77122 has finished for PR 17298 at commit c450d24.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #77127 has finished for PR 17298 at commit e0c3a6b.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #77151 has finished for PR 17298 at commit 009e7c4.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk

Copy link
Copy Markdown
ContributorAuthor

Jenkins retest this please.

@SparkQA

Copy link
Copy Markdown

Test build #82050 has finished for PR 17298 at commit 009e7c4.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk

Copy link
Copy Markdown
ContributorAuthor

cc @BryanCutler@HyukjinKwon

@SparkQA

Copy link
Copy Markdown

Test build #86233 has finished for PR 17298 at commit ae450e8.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #86238 has finished for PR 17298 at commit 33a2bab.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk

Copy link
Copy Markdown
ContributorAuthor

Eh we don't seem to need this anymore we plumb through the exception messages in another way. it's missing the logs but it's probably not worth it. Closing.

@holdenkholdenk closed this Jan 18, 2018
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@holdenk@SparkQA