Uh oh!
There was an error while loading. Please reload this page.
Add tests for FileLogger, EventLoggingListener, and ReplayListenerBus - #591
Closed
andrewor14 wants to merge 14 commits into
Closed
Add tests for FileLogger, EventLoggingListener, and ReplayListenerBus#591andrewor14 wants to merge 14 commits into
andrewor14 wants to merge 14 commits into
Conversation
An event can be mutated by the DAGScheduler in between being procssed by one listener and being processed by another. This causes the ReplayListenerSuite to be flaky. This commit ensures that the event logged is the same as the original event received by the EventLoggingListener.
Conflicts: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala core/src/main/scala/org/apache/spark/util/FileLogger.scala
AmplabJenkins
commented
Apr 29, 2014
Merged build triggered. |
AmplabJenkins
commented
Apr 29, 2014
Merged build started. |
AmplabJenkins
commented
Apr 29, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
Apr 29, 2014
All automated tests passed. |
AmplabJenkins
commented
Apr 30, 2014
Merged build triggered. |
AmplabJenkins
commented
Apr 30, 2014
Merged build started. |
AmplabJenkins
commented
Apr 30, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
Apr 30, 2014
All automated tests passed. |
Contributor
There was a problem hiding this comment.
I don't think this was over the line limit before (?) wouldn't it have failed our style checks?
ContributorAuthor
There was a problem hiding this comment.
I renamed it to sparkConf, because there's also a hadoopConf now
Contributor
There was a problem hiding this comment.
Ah I see - I misread this to only be a linebreak change
pwendell
commented
Apr 30, 2014
Contributor
Unfortunately I only had time to do a cursory glance. I mostly just sanity checked the changes to the non-test code to make sure there were no mistakes. Looks good to me pending some small comments about the tests. |
AmplabJenkins
commented
May 1, 2014
Build triggered. |
AmplabJenkins
commented
May 1, 2014
Build started. |
Conflicts: core/src/main/scala/org/apache/spark/util/Utils.scala
AmplabJenkins
commented
May 1, 2014
Merged build triggered. |
AmplabJenkins
commented
May 1, 2014
Merged build started. |
AmplabJenkins
commented
May 1, 2014
Build finished. |
AmplabJenkins
commented
May 1, 2014
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14597/ |
AmplabJenkins
commented
May 1, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
May 1, 2014
All automated tests passed. |
pwendell
commented
May 2, 2014
Contributor
Thanks - I merged this. |
asfgit pushed a commit
that referenced
this pull request
May 2, 2014
Modifications to Spark core are limited to exposing functionality to test files + minor style fixes. (728 / 769 lines are from tests) Author: Andrew Or <andrewor14@gmail.com> Closes#591 from andrewor14/event-log-tests and squashes the following commits: 2883837 [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests c3afcea [Andrew Or] Compromise 2d5daf8 [Andrew Or] Use temp directory provided by the OS rather than /tmp 2b52151 [Andrew Or] Remove unnecessary file delete + add a comment 62010fd [Andrew Or] More cleanup (renaming variables, updating comments etc) ad2beff [Andrew Or] Clean up EventLoggingListenerSuite + modify a few comments 862e752 [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests e0ba2f8 [Andrew Or] Fix test failures caused by race condition in processing/mutating events b990453 [Andrew Or] ReplayListenerBus suite - tests do not all pass yet ab66a84 [Andrew Or] Tests for FileLogger + delete file after tests 187bb25 [Andrew Or] Formatting and renaming variables 769336f [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests 5d38ffe [Andrew Or] Clean up EventLoggingListenerSuite + add comments e12f4b1 [Andrew Or] Preliminary tests for EventLoggingListener (need major cleanup) (cherry picked from commit 394d8cb) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
pwendell pushed a commit
to pwendell/spark
that referenced
this pull request
May 12, 2014
SPARK-1076: [Fixapache#578] add @transient to some vals I'll try to be more careful next time. Author: Xiangrui Meng <meng@databricks.com> Closesapache#591 and squashes the following commits: 2b4f044 [Xiangrui Meng] add @transient to prev in ZippedWithIndexRDD add @transient to seed in PartitionwiseSampledRDD
pdeyhim pushed a commit
to pdeyhim/spark-1
that referenced
this pull request
Jun 25, 2014
Modifications to Spark core are limited to exposing functionality to test files + minor style fixes. (728 / 769 lines are from tests) Author: Andrew Or <andrewor14@gmail.com> Closesapache#591 from andrewor14/event-log-tests and squashes the following commits: 2883837 [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests c3afcea [Andrew Or] Compromise 2d5daf8 [Andrew Or] Use temp directory provided by the OS rather than /tmp 2b52151 [Andrew Or] Remove unnecessary file delete + add a comment 62010fd [Andrew Or] More cleanup (renaming variables, updating comments etc) ad2beff [Andrew Or] Clean up EventLoggingListenerSuite + modify a few comments 862e752 [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests e0ba2f8 [Andrew Or] Fix test failures caused by race condition in processing/mutating events b990453 [Andrew Or] ReplayListenerBus suite - tests do not all pass yet ab66a84 [Andrew Or] Tests for FileLogger + delete file after tests 187bb25 [Andrew Or] Formatting and renaming variables 769336f [Andrew Or] Merge branch 'master' of github.com:apache/spark into event-log-tests 5d38ffe [Andrew Or] Clean up EventLoggingListenerSuite + add comments e12f4b1 [Andrew Or] Preliminary tests for EventLoggingListener (need major cleanup)
gzm55 pushed a commit
to MediaV/spark
that referenced
this pull request
Jul 17, 2014
SPARK-1076: [Fixapache#578] add @transient to some vals I'll try to be more careful next time. Author: Xiangrui Meng <meng@databricks.com> Closesapache#591 and squashes the following commits: 2b4f044 [Xiangrui Meng] add @transient to prev in ZippedWithIndexRDD add @transient to seed in PartitionwiseSampledRDD Conflicts: core/src/main/scala/org/apache/spark/rdd/PartitionwiseSampledRDD.scala
bzhaoopenstack pushed a commit
to bzhaoopenstack/spark
that referenced
this pull request
Sep 11, 2019
update `status-url` to the specified place.
RolatZhang pushed a commit
to RolatZhang/spark
that referenced
this pull request
Feb 24, 2023
…e#591) * Revert "Revert "[SPARK-37600][BUILD] Upgrade to Hadoop 3.3.2" (apache#565)" This reverts commit 197dc34. * Revert "minor fix ut" This reverts commit 6ef82b4.
RolatZhang pushed a commit
to RolatZhang/spark
that referenced
this pull request
Dec 8, 2023
…e#591) * Revert "Revert "[SPARK-37600][BUILD] Upgrade to Hadoop 3.3.2" (apache#565)" This reverts commit 197dc34. * Revert "minor fix ut" This reverts commit 6ef82b4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Modifications to Spark core are limited to exposing functionality to test files + minor style fixes.
(728 / 769 lines are from tests)