Skip to content

[SparkR][Doc] fix typo in vignettes - #17884

Closed
actuaryzhang wants to merge 3 commits into
apache:masterfrom
actuaryzhang:typo
Closed

[SparkR][Doc] fix typo in vignettes#17884
actuaryzhang wants to merge 3 commits into
apache:masterfrom
actuaryzhang:typo

Conversation

@actuaryzhang

@actuaryzhangactuaryzhang commented May 6, 2017

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Fix typo in vignettes

@actuaryzhang

actuaryzhang commented May 6, 2017

Copy link
Copy Markdown
ContributorAuthor

@felixcheung

@SparkQA

SparkQA commented May 6, 2017

Copy link
Copy Markdown

Test build #76527 has finished for PR 17884 at commit 8639025.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

HyukjinKwon commented May 6, 2017

Copy link
Copy Markdown
Member

I know it is legitimate but It would be worth double checking other typos too. Usually, single typo PR is not encouraged up to my knowledge given reviwing, building and merging costs.

@actuaryzhang

actuaryzhang commented May 6, 2017

Copy link
Copy Markdown
ContributorAuthor

@HyukjinKwon Thanks for pointing this out. I will keep this in mind next time.

@felixcheung

felixcheung commented May 7, 2017

Copy link
Copy Markdown
Member

This test seems flaky on AppVeyor, not sure why

Failed -------------------------------------------------------------------------
1. Error: spark.glm and predict (@test_mllib_regression.R#57) ------------------
java.lang.IllegalStateException: SparkContext has been shutdown
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2015)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2044)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2063)
at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:333)
at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:38)
at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collectFromPlan(Dataset.scala:2923)
at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2237)
at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2237)
at org.apache.spark.sql.Dataset$$anonfun$57.apply(Dataset.scala:2907)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:2906)
at org.apache.spark.sql.Dataset.head(Dataset.scala:2237)
at org.apache.spark.sql.Dataset.head(Dataset.scala:2244)
at org.apache.spark.sql.Dataset.first(Dataset.scala:2251)

@felixcheung

felixcheung commented May 7, 2017

Copy link
Copy Markdown
Member

@actuaryzhang thanks - would you have a chance to run a quick QA check on the rest of the vignettes, if you haven't already?

@actuaryzhang

actuaryzhang commented May 7, 2017

Copy link
Copy Markdown
ContributorAuthor

@felixcheung I ran a quick QA on the vignettes and fixed some additional typos and styles.

@SparkQA

SparkQA commented May 7, 2017

Copy link
Copy Markdown

Test build #76534 has finished for PR 17884 at commit 796a8e7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

felixcheung
felixcheung approved these changes May 7, 2017
View reviewed changes

@felixcheungfelixcheung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, thanks! just one minor request

Comment threadR/pkg/vignettes/sparkr-vignettes.Rmd Outdated
```

Like gapply, `gapplyCollect` applies a function to each partition of a `SparkDataFrame` and collect the result back to R `data.frame`. The output of the function should be a `data.frame` but no schema is required in this case. Note that `gapplyCollect` can fail if the output of UDF run on all the partition cannot be pulled to the driver and fit in driver memory.
Like gapply, `gapplyCollect` applies a function to each partition of a `SparkDataFrame` and collect the result back to R `data.frame`. The output of the function should be a `data.frame` but no schema is required in this case. Note that `gapplyCollect` can fail if the output of the UDF on all partitions cannot be pulled into the driver's memory.

@felixcheungfelixcheungMay 7, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add backtick to gapply at the beginning

@actuaryzhangactuaryzhangMay 7, 2017

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment threadR/pkg/vignettes/sparkr-vignettes.Rmd
+ `env` saves the meta-information of the object such as `isCached`.

It can be created by data import methods or by transforming an existing `SparkDataFrame`. We can manipulate `SparkDataFrame` by numerous data processing functions and feed that into machine learning algorithms.
It can be created by data import methods or by transforming an existing `SparkDataFrame`. We can manipulate `SparkDataFrame` by numerous data processing functions and feed that into machine learning algorithms.

@felixcheungfelixcheungMay 7, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, does this whitespace in front of paragraph get handled properly?

@actuaryzhangactuaryzhangMay 7, 2017•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@felixcheung Yes, the four spaces indicate that the text following should be aligned with the bullet point. Otherwise, it will start as a new paragraph and have the wrong indention.
You will see the difference after compiling the Rmarkdown file.

@felixcheungfelixcheungMay 7, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool! thanks

@SparkQA

SparkQA commented May 7, 2017

Copy link
Copy Markdown

Test build #76555 has finished for PR 17884 at commit b0407b5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

asfgit pushed a commit that referenced this pull request May 8, 2017
Wayne ZhangFelix Cheung
[SPARKR][DOC] fix typo in vignettes
d8a5a0d
## What changes were proposed in this pull request?
Fix typo in vignettes
Author: Wayne Zhang <actuaryzhang@uber.com>
Closes#17884 from actuaryzhang/typo.
(cherry picked from commit 2fdaeb5)
Signed-off-by: Felix Cheung <felixcheung@apache.org>
@asfgitasfgit closed this in 2fdaeb5May 8, 2017
@felixcheung

felixcheung commented May 8, 2017

Copy link
Copy Markdown
Member

merged to master/2.2
thanks!

@actuaryzhang
actuaryzhang deleted the typo branch May 8, 2017 06:22
lycplus pushed a commit to lycplus/spark that referenced this pull request May 24, 2017
Wayne Zhang@lycplus
[SPARKR][DOC] fix typo in vignettes
1055955
## What changes were proposed in this pull request?
Fix typo in vignettes
Author: Wayne Zhang <actuaryzhang@uber.com>
Closesapache#17884 from actuaryzhang/typo.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixcheungfelixcheungfelixcheung approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@actuaryzhang@SparkQA@HyukjinKwon@felixcheung