feat!: Require parentheses around pipelines

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error

[max-sixty]

Whether pipelines required parentheses was ambiguous before — does select (invoice_date | date.to_text "%d/%m/%Y") require its parentheses? How about in a tuple like select {(invoice_date | date.to_text "%d/%m/%Y")}?

This changes the parser so parentheses are required. This creates much better errors in some cases — for example a missing comma from a tuple isn't parsed as a pipeline:

derive {
x=y
a=b
}

...is no longer parsed as derive {x=(y | a=b)}, which is really confusing. Or similarly derive {x=y a=b} also parses into that.

It also simplifies the parser code slightly; unifying pipeline functions.

(tests fail on parsing one thing from the std lib, I need to fix)

[snth]

It does make a lot of sense for those parsing error examples that you show. It's a pity though for the common use cases like sum or text.lower, e.g. derive low = (title | text.lower), where it adds overhead.

Would it be possible to require it only in cases where there's a \n involved? In Python you have

>>>s='a''b'>>>s'ab'>>>s= ('a'
... 'b')
>>>s'ab'

[snth]

Or if one conceptually separated the two uses of |, maybe one could disallow \n as composition operator for the expression language?

What I mean is, assume one used |> or >> for piping relations to transforms and | for piping arguments to functions in column expressions, e.g. from customers >> derive upper_title = title | text.upper. While it's sometimes useful to use >> as above, we usually write this as

from customers
derive upper_title = title | text.upper

While I usually want to use \n instead of >>, I struggle to imagine scenarios where I would want to use \n instead of |. Even in cases where the line becomes very long and I want to split it over two lines, I'd be ok with requiring a mandatory | as well as () to suppress the conversion of \n to >>. So

from customers
derive upper_title =(title |...|...|
text.upper)

or

from customers
derive upper_title =(title | text.upper)

but never

from customers
derive upper_title =(title text.upper)

or

from customers
derive upper_title = title text.upper

[max-sixty]

I think any deviation away from "| is the same as a newline in a pipeline" is a huge change and would need a really compelling rationale. Similarly anything making a pipeline anything other than applying functions in turn...

[max-sixty]

Need to add some parentheses docs but otherwise this is ready to go!

Edit: actually the docs were already up-to-date... the code behavior was overly permissive and we didn't check for an error