Uh oh!
There was an error while loading. Please reload this page.
ARROW-12995: [C++] Add validation to CSV options - #10505
Conversation
7ab0c8d to
5dc61deCompareThere was a problem hiding this comment.
Hmm, I don't think I understand this check. Is there something in the CSV parser that currently prevents using other delimiters?
There was a problem hiding this comment.
A lot of the characters before are terminal control characters and the only ascii character after ~ is del. While these characters could be used in theory for a text file they would be very strange couldn't be displayed on a terminal. These are the characters this is excluding https://flaviocopes.com/non-printable-ascii-characters/ . Also, I don't think the user could specify values like \n or \r since that would break the parser since they have to be used for end of line. Currently the python layer requires all character values to be between 1 - 127 inclusive.
I kept tab because tab separated values is a format which is used.
There was a problem hiding this comment.
I don't think it's our duty to guard against dubious values. Perhaps some weird formats use a form-feed character as a delimiter, who knows?
The only think that may be reasonable may be to forbid \n and \r. Otherwise we should just let the user choose whatever they like.
There was a problem hiding this comment.
Ooops, I wanted 1KB but tripped over tests. Guess I missed a spot
cdda435 to
92726b7Compare92726b7 to
942d5f7Comparepitrou
commented
Jun 16, 2021
Rebased, will merge if CI is green. |
pitrou
commented
Jun 16, 2021
AppVeyor failure is unrelated (https://issues.apache.org/jira/browse/ARROW-13090). |
pitrou
commented
Jun 16, 2021
Thank you again @n3world ! |
Closesapache#10505 from n3world/ARROW-12995-Validate_csv_opts Authored-by: Nate Clark <nate@neworld.us> Signed-off-by: Antoine Pitrou <antoine@python.org>
No description provided.