[SPARK-22323] Design doc for pandas_udf

[icexelloss]

I open this PR so we can have a place to discuss the design.

We don't necessary need to merge a md file for the doc - this be can embeded python documentation.

[icexelloss]

cc @cloud-fan@ueshin@HyukjinKwon@gatorsmile@viirya

To continue the discussion on #19505

[SparkQA]

Test build #82935 has finished for PR 19544 at commit 3005312.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

@jiangxb1987 will reorg the existing Spark SQL doc. We can think about how to put this into the new version of Spark SQL doc.

[srowen]

I don't think we would merge this to Spark docs right? I don't think this is an efficient way to discuss a design document if that's the intent

[gatorsmile]

This is not ready yet. Eventually, we need a document for vectorized Python UDF

[icexelloss]

Sorry for the late reply.

@srowen At this point I would like to have feedback on the design doc so we can move forward with implementation or a formal Spark docs if people prefer. What do you think is a best way to discuss design document?

[icexelloss]

Do people prefer google doc? I can do that too.

[cloud-fan]

I feel google doc is easier to review and discuss

[icexelloss]

Ok. I will put in a google doc and ping your guys. Thanks!

…

On Wed, Oct 25, 2017 at 11:25 AM Wenchen Fan ***@***.***> wrote: I feel google doc is easier to review and discuss — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19544 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAwbrIqz5Uyp7DJXOI8Ofq_GyJTs1MEIks5sv1LugaJpZM4QA0na> .

[icexelloss]

Thanks all for the advice. I've put this in a google doc.

https://docs.google.com/document/d/1KlLaa-xJ3oz28xlEJqXyCAHU3dwFYkFs_ixcUXrJNTc/edit?usp=sharing

[icexelloss]

Close this in favor of the google doc.