Redshift: Fix parsing for quoted numbered columns

[7phs]

Quoted identifiers allow the use of any characters and can start with any character, including digits. For example:

SELECT1AS"1"FROM a

However, the tokenizer cannot parse such types of quoted identifiers in the case of Redshift. This issue is related to the handling of potential identifiers in PartiQL.

This merge request (MR) clarifies the behavior of quoted identifiers in PartiQL and fixes the parsing of quoted identifiers for Redshift.

I added an additional case of a valid JSON path in a specific unit test to ensure that JSON path parsing is not broken.

[7phs]

This code is RedShift specific (nested quotation). The same time tokenisation of quoted identifier is a method of tokeniser and this is a reason of implemented it here.

Let me know if you have a good idea how makes this code more explicit.

[iffyio]

Thanks @7phs! Left some minor comments I think the approach looks reasonable!

[bombsimon]

🚀

[iffyio]

LGTM! Thanks @7phs@bombsimon
cc @alamb