Uh oh!
There was an error while loading. Please reload this page.
[SPARK-782] Made Spark use existing shaded ASM and removed Spark's ASM dependency - #232
Closed
tdas wants to merge 1 commit into
Closed
[SPARK-782] Made Spark use existing shaded ASM and removed Spark's ASM dependency#232tdas wants to merge 1 commit into
tdas wants to merge 1 commit into
Conversation
tdas
commented
Mar 26, 2014
ContributorAuthor
AmplabJenkins
commented
Mar 26, 2014
Merged build triggered. |
AmplabJenkins
commented
Mar 26, 2014
Merged build started. |
pwendell
commented
Mar 26, 2014
Contributor
Looks good to me. Thanks TD! |
AmplabJenkins
commented
Mar 26, 2014
Merged build finished. |
AmplabJenkins
commented
Mar 26, 2014
All automated tests passed. |
mridulm
commented
Mar 26, 2014
Contributor
Which dependency is providing this shaded version ? |
pwendell
commented
Mar 26, 2014
Contributor
We're using the shaded version from Kryo. If they change this in future releases we can change accordingly, but other projects are also using Kryo's shaded version so I don't think it's something they are likely to change. |
tdas
commented
Mar 26, 2014
ContributorAuthor
Also, refer to the discussion here #100 This PR only backports the changes already accepted in master. |
asfgit pushed a commit
that referenced
this pull request
Mar 26, 2014
…M dependency This ports the changes in #100 to branch 0.9. However, unlike that PR, it does not exclude ASM from all dependencies of Spark, to ensure compatibility in branch 0.9. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes#232 from tdas/asm and squashes the following commits: 999bb6f [Tathagata Das] Made Spark use existing shaded ASM and removed Spark's ASM depedency.
jhartlaub referenced
this pull request
in jhartlaub/spark
May 27, 2014
jobWaiter.synchronized before jobWaiter.wait ...else ``IllegalMonitorStateException`` in ``SimpleFutureAction#ready``. (cherry picked from commit 0780498) Signed-off-by: Reynold Xin <rxin@apache.org>
jhartlaub referenced
this pull request
in jhartlaub/spark
May 27, 2014
jobWaiter.synchronized before jobWaiter.wait ...else ``IllegalMonitorStateException`` in ``SimpleFutureAction#ready``. (cherry picked from commit 0780498) Signed-off-by: Reynold Xin <rxin@apache.org>
asfgit pushed a commit
that referenced
this pull request
Nov 11, 2015
…mpiled classes This patch modifies Spark's closure cleaner (and a few other places) to use ASM 5, which is necessary in order to support cleaning of closures that were compiled by Java 8. In order to avoid ASM dependency conflicts, Spark excludes ASM from all of its dependencies and uses a shaded version of ASM 4 that comes from `reflectasm` (see [SPARK-782](https://issues.apache.org/jira/browse/SPARK-782) and #232). This patch updates Spark to use a shaded version of ASM 5.0.4 that was published by the Apache XBean project; the POM used to create the shaded artifact can be found at https://github.com/apache/geronimo-xbean/blob/xbean-4.4/xbean-asm5-shaded/pom.xml. http://movingfulcrum.tumblr.com/post/80826553604/asm-framework-50-the-missing-migration-guide was a useful resource while upgrading the code to use the new ASM5 opcodes. I also added a new regression tests in the `java8-tests` subproject; the existing tests were insufficient to catch this bug, which only affected Scala 2.11 user code which was compiled targeting Java 8. Author: Josh Rosen <joshrosen@databricks.com> Closes#9512 from JoshRosen/SPARK-6152.
asfgit pushed a commit
that referenced
this pull request
Nov 11, 2015
…mpiled classes This patch modifies Spark's closure cleaner (and a few other places) to use ASM 5, which is necessary in order to support cleaning of closures that were compiled by Java 8. In order to avoid ASM dependency conflicts, Spark excludes ASM from all of its dependencies and uses a shaded version of ASM 4 that comes from `reflectasm` (see [SPARK-782](https://issues.apache.org/jira/browse/SPARK-782) and #232). This patch updates Spark to use a shaded version of ASM 5.0.4 that was published by the Apache XBean project; the POM used to create the shaded artifact can be found at https://github.com/apache/geronimo-xbean/blob/xbean-4.4/xbean-asm5-shaded/pom.xml. http://movingfulcrum.tumblr.com/post/80826553604/asm-framework-50-the-missing-migration-guide was a useful resource while upgrading the code to use the new ASM5 opcodes. I also added a new regression tests in the `java8-tests` subproject; the existing tests were insufficient to catch this bug, which only affected Scala 2.11 user code which was compiled targeting Java 8. Author: Josh Rosen <joshrosen@databricks.com> Closes#9512 from JoshRosen/SPARK-6152. (cherry picked from commit 529a1d3) Signed-off-by: Michael Armbrust <michael@databricks.com>
dskrvk pushed a commit
to dskrvk/spark
that referenced
this pull request
Nov 13, 2015
…mpiled classes This patch modifies Spark's closure cleaner (and a few other places) to use ASM 5, which is necessary in order to support cleaning of closures that were compiled by Java 8. In order to avoid ASM dependency conflicts, Spark excludes ASM from all of its dependencies and uses a shaded version of ASM 4 that comes from `reflectasm` (see [SPARK-782](https://issues.apache.org/jira/browse/SPARK-782) and apache#232). This patch updates Spark to use a shaded version of ASM 5.0.4 that was published by the Apache XBean project; the POM used to create the shaded artifact can be found at https://github.com/apache/geronimo-xbean/blob/xbean-4.4/xbean-asm5-shaded/pom.xml. http://movingfulcrum.tumblr.com/post/80826553604/asm-framework-50-the-missing-migration-guide was a useful resource while upgrading the code to use the new ASM5 opcodes. I also added a new regression tests in the `java8-tests` subproject; the existing tests were insufficient to catch this bug, which only affected Scala 2.11 user code which was compiled targeting Java 8. Author: Josh Rosen <joshrosen@databricks.com> Closesapache#9512 from JoshRosen/SPARK-6152.
liancheng pushed a commit
to liancheng/spark
that referenced
this pull request
Mar 17, 2017
…rotocol ## What changes were proposed in this pull request? This avoids being latency bound on file reads. Initial testing shows this to be at least 5x faster than without the thread pool. ## How was this patch tested? Existing tests; perf test on dogfood. Author: Eric Liang <ekl@databricks.com> Closesapache#232 from ericl/sc-5937.
Igosuki pushed a commit
to Adikteev/spark
that referenced
this pull request
Jul 31, 2018
bzhaoopenstack pushed a commit
to bzhaoopenstack/spark
that referenced
this pull request
Sep 11, 2019
Upload multiple junit files to avoid test timeout
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This ports the changes in #100 to branch 0.9. However, unlike that PR, it does not exclude ASM from all dependencies of Spark, to ensure compatibility in branch 0.9.