[SPARK-8238][SPARK-8239][SPARK-8242][SPARK-8243][SPARK-8268][SQL]Add ascii/base64/unbase64/encode/decode functions by chenghao-intel · Pull Request #6843 · apache/spark

chenghao-intel · 2015-06-16T14:36:42Z

Add ascii,base64,unbase64,encode and decode expressions.

SparkQA · 2015-06-16T14:41:24Z

Test build #34986 has finished for PR 6843 at commit b341b21.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Base64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Decode(bin: Expression, charset: Expression)
- case class Encode(value: Expression, charset: Expression)

chenghao-intel · 2015-06-16T15:16:49Z

@adrian-wang @zhichao-li can you review the code for me?

SparkQA · 2015-06-16T17:10:12Z

Test build #34990 has finished for PR 6843 at commit 6a71481.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Base64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Decode(bin: Expression, charset: Expression)
- case class Encode(value: Expression, charset: Expression)

adrian-wang · 2015-06-17T02:57:30Z

Upper case these all?

SparkQA · 2015-06-18T15:24:41Z

Test build #35132 has finished for PR 6843 at commit 05cc18e.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression
- case class Base64(child: Expression) extends UnaryExpression
- case class UnBase64(child: Expression) extends UnaryExpression
- case class Decode(bin: Expression, charset: Expression) extends Expression
- case class Encode(value: Expression, charset: Expression) extends Expression

chenghao-intel · 2015-06-19T00:36:46Z

@rxin, any more comments on this?

rxin · 2015-06-19T05:31:20Z

i don't think this is necessary since it is a case class?

btw i don't think we should be making everything uppercase..

Probably the registered function name will make more sense, than giving the user a class name. What do you think?

SparkQA · 2015-06-23T08:19:33Z

Test build #35522 has finished for PR 6843 at commit a5e89b6.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression
- case class Base64(child: Expression) extends UnaryExpression
- case class UnBase64(child: Expression) extends UnaryExpression
- case class Decode(bin: Expression, charset: Expression) extends Expression
- case class Encode(value: Expression, charset: Expression) extends Expression

SparkQA · 2015-07-02T04:54:37Z

Test build #36330 has finished for PR 6843 at commit da6a788.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class Base64(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class Decode(bin: Expression, charset: Expression)
- case class Encode(value: Expression, charset: Expression)

SparkQA · 2015-07-02T06:59:03Z

Test build #36341 has finished for PR 6843 at commit 947a88a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- trait ExpectsInputTypes
- trait AutoCastInputTypes
- abstract class BinaryExpression extends Expression with trees.BinaryNode[Expression]
- abstract class BinaryOperator extends BinaryExpression
- abstract class BinaryArithmetic extends BinaryOperator
- abstract class BinaryComparison extends BinaryOperator with Predicate
- case class Ascii(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class Base64(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with AutoCastInputTypes
- case class Decode(bin: Expression, charset: Expression)
- case class Encode(value: Expression, charset: Expression)

SparkQA · 2015-07-03T15:19:29Z

Test build #36500 has finished for PR 6843 at commit 9d6f9f4.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Factorial(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Ascii(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Base64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Decode(bin: Expression, charset: Expression) extends Expression with ExpectsInputTypes
- case class Encode(value: Expression, charset: Expression)

SparkQA · 2015-07-03T15:24:17Z

Test build #36501 has finished for PR 6843 at commit 78dee7d.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Ascii(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Base64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class UnBase64(child: Expression) extends UnaryExpression with ExpectsInputTypes
- case class Decode(bin: Expression, charset: Expression) extends Expression with ExpectsInputTypes
- case class Encode(value: Expression, charset: Expression)

rxin · 2015-07-04T06:39:11Z

can you make this extend BinaryExpression? You can just define def bin = left, and def charset = right.

Actually that's my intention, as I think the parameters is asymmetric semantically. Not sure if you are thinking the code impovement like #7157?

chenghao-intel changed the title ~~[SPARK-8238][SPARK-8239][SPARK-8242][SPARK-8243][SPARK-8268][SQL]add ascii/base64/unbase64/encode/decode functions~~[SPARK-8238][SPARK-8239][SPARK-8242][SPARK-8243][SPARK-8268][SQL][WIP] Add ascii/base64/unbase64/encode/decode functionsJun 16, 2015

adrian-wang reviewed Jun 17, 2015
View reviewed changes

chenghao-intel changed the title ~~[SPARK-8238][SPARK-8239][SPARK-8242][SPARK-8243][SPARK-8268][SQL][WIP] Add ascii/base64/unbase64/encode/decode functions~~[SPARK-8238][SPARK-8239][SPARK-8242][SPARK-8243][SPARK-8268][SQL]Add ascii/base64/unbase64/encode/decode functionsJun 18, 2015

rxin reviewed Jun 19, 2015
View reviewed changes

chenghao-intel force-pushed the str_funcs2 branch from a5e89b6 to da6a788Compare July 2, 2015 03:11

chenghao-intel force-pushed the str_funcs2 branch from da6a788 to 947a88aCompare July 2, 2015 05:00

chenghao-intel added 5 commits July 3, 2015 21:26

add ascii/base64/unbase64/encode/decode functions
491ce7b

remove the unused import
e2df768

scalastyle issues
96170fc

update code as comments
ed5c19c

remove the toString method for expressions
9d6f9f4

chenghao-intel force-pushed the str_funcs2 branch from 947a88a to 9d6f9f4Compare July 3, 2015 13:42

base 64 -> base64
78dee7d

rxin reviewed Jul 4, 2015
View reviewed changes

Uh oh!

Conversation

chenghao-intel commented Jun 16, 2015

Uh oh!

SparkQA commented Jun 16, 2015

Uh oh!

chenghao-intel commented Jun 16, 2015

Uh oh!

SparkQA commented Jun 16, 2015

Uh oh!

adrian-wangJun 17, 2015

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jun 18, 2015

Uh oh!

chenghao-intel commented Jun 19, 2015

Uh oh!

rxinJun 19, 2015

Choose a reason for hiding this comment

Uh oh!

rxinJun 19, 2015

Choose a reason for hiding this comment

Uh oh!

chenghao-intelJun 19, 2015

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jun 23, 2015

Uh oh!

SparkQA commented Jul 2, 2015

Uh oh!

SparkQA commented Jul 2, 2015

Uh oh!

SparkQA commented Jul 3, 2015

Uh oh!

SparkQA commented Jul 3, 2015

Uh oh!

rxinJul 4, 2015

Choose a reason for hiding this comment

Uh oh!

chenghao-intelJul 5, 2015

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants