fix: bind an r-string's closing quote to its opening quote

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.

[prql-bot]

Problem

raw_string in the lexer parses the opening and closing quotes as two independent choice((just('\''), just('"'))) branches and discards both, so nothing requires them to match. Two consequences, both confirmed against main at 7dc4962:

• from t | select {a = r"hello'} compiles to SELECT 'hello' AS a FROM t — a mismatched delimiter silently closes the string instead of erroring.
• The content filter excludes both quote characters, so r"it's" and r'say "hi"' fail to lex, even though the delimiter is unambiguous.

Solution

Parameterize the parser on the opening quote (delimited_by('"') / delimited_by('\'')) and require the same character to close it. The content filter then excludes only the delimiter, which is how multi_quoted_string already handles regular strings. r"hello' now reports an unterminated string, and r"it's" lexes as the four characters it's.

Testing

New raw_string_delimiters test in prqlc/prqlc-parser/src/lexer/test.rs covers both directions — it fails on main (the mismatched-delimiter cases lex successfully there) and passes with the fix. cargo test -p prqlc-parser, -p prqlc and -p mdbook-prql are green locally; the book gains a short r-strings example for the newly-accepted form, with its snapshot.

Found during the nightly rolling survey of prqlc/prqlc-parser/src/lexer/mod.rs.