Skip to content

[SPARK-20754][SQL][FOLLOWUP] Add Function Alias For MOD/POSITION. - #18206

Closed
wangyum wants to merge 4 commits into
apache:masterfrom
wangyum:SPARK-20754-mod&position
Closed

[SPARK-20754][SQL][FOLLOWUP] Add Function Alias For MOD/POSITION.#18206
wangyum wants to merge 4 commits into
apache:masterfrom
wangyum:SPARK-20754-mod&position

Conversation

@wangyum

@wangyumwangyum commented Jun 5, 2017

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

#18106 Support TRUNC (number), We should also add function alias for MOD and POSITION.

POSITION(substr IN str) is a synonym for LOCATE(substr,str). same as MySQL: https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_position

How was this patch tested?

unit tests

@SparkQA

Copy link
Copy Markdown

Test build #77748 has finished for PR 18206 at commit 6168d3f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile

Copy link
Copy Markdown
Member

@wangyum Could you resolve the conflicts? Thanks!

select length(uuid()), (uuid() <> uuid());

-- position
select position('bar', 'foobarbar'), position('bar', 'foobarbar', 5), position(null, 'foobarbar'), position('aaads', null);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The syntax of position is POSITION(substr IN str). It is different from LOCATE. You need to change the parser to support it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check what we did for remainder and the others in Parser supports.

select floor(1234567890123456);

-- mod
select mod(7, 2), mod(7, 0), mod(0, 2), mod(7, null), mod(null, 2), mod(null, null);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update the function description of Remainder . The example does not show this syntax.

…ition
Conflicts:
sql/core/src/test/resources/sql-tests/inputs/operators.sql
sql/core/src/test/resources/sql-tests/results/operators.sql.out
@SparkQA

Copy link
Copy Markdown

Test build #77993 has finished for PR 18206 at commit 8cce02b.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmilegatorsmile left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good. Left a few minor comments.

| SET | RESET
| VIEW | REPLACE
| IF
| POSITION

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the keyword to TableIdentifierParserSuite

| identifier #columnReference
| base=primaryExpression '.' fieldName=identifier #dereference
| '(' expression ')' #parenthesizedExpression
| POSITION '(' valueExpression IN valueExpression ')' #position

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move it to the line 566

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

POSITION '(' substr=valueExpression IN str=valueExpression ')'

@@ -851,6 +853,8 @@ IGNORE: 'IGNORE';

IF: 'IF';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this space.

4
> SELECT _FUNC_('bar', 'foobarbar', 5);
7
> SELECT POSITION('bar' in 'foobarbar');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in -> IN

/**
* Create a Position expression.
*/
override def visitPosition(ctx: PositionContext): Expression = withOrigin(ctx) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new StringLocate(expression(ctx.substr), expression(ctx.str))

@SparkQA

Copy link
Copy Markdown

Test build #78014 has finished for PR 18206 at commit d068604.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile

Copy link
Copy Markdown
Member

LGTM

@gatorsmile

Copy link
Copy Markdown
Member

Thanks! Merging to master.

dataknocker pushed a commit to dataknocker/spark that referenced this pull request Jun 16, 2017
## What changes were proposed in this pull request?
apache#18106 Support TRUNC (number), We should also add function alias for `MOD `and `POSITION`.
`POSITION(substr IN str) `is a synonym for `LOCATE(substr,str)`. same as MySQL: https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_position
## How was this patch tested?
unit tests
Author: Yuming Wang <wgyumg@gmail.com>
Closesapache#18206 from wangyum/SPARK-20754-mod&position.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wangyum@SparkQA@gatorsmile