Undocumented behavior in csv.Sniffer (preferred delimiters)

[ehrmann]

BPO	36497
Nosy	@avinassh, @ehrmann

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-04-01.13:54:16.454>
labels = ['3.7', '3.8', 'type-bug', 'library', 'docs']
title = 'Undocumented behavior in csv.Sniffer (preferred delimiters)'
updated_at = <Date 2019-06-30.12:23:03.098>
user = 'https://github.com/ehrmann'

bugs.python.org fields:

activity = <Date 2019-06-30.12:23:03.098>
actor = 'avi'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Library (Lib)']
creation = <Date 2019-04-01.13:54:16.454>
creator = 'thomas'
dependencies = []
files = []
hgrepos = []
issue_num = 36497
keywords = []
message_count = 2.0
messages = ['339291', '346926']
nosy_count = 3.0
nosy_names = ['docs@python', 'avi', 'thomas']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue36497'
versions = ['Python 3.7', 'Python 3.8']

Linked PRs

• gh-80678: Add comment on preferred csv delimiters #96199
• gh-80678: Document the preferred delimiters of csv.Sniffer #154336
• [3.15] gh-80678: Document the preferred delimiters of csv.Sniffer (GH-154336) #154589
• [3.14] gh-80678: Document the preferred delimiters of csv.Sniffer (GH-154336) #154590
• [3.13] gh-80678: Document the preferred delimiters of csv.Sniffer (GH-154336) #154591
• gh-80678: Document the preferred attribute of csv.Sniffer #155068

[ehrmann]

When the Sniffer detects more than one possible delimiter, as e.g. in the following file

"a;b;c;d,e;f;g;h"

the result will always be the ',' delimiter, independent of how "dominant" another delimiter is. This is because the codepath analyzing dominance will only get executed if the undocumented Sniffer member Sniffer.preferred is overwritten by the user after initialization.

While not strictly a bug, the behavior should probably be documented, and the 'preferred' member could be exposed as an argument in __init__() perhaps?

[avinassh]

can I take up this issue?

[serhiy-storchaka]

The behavior is still the same after the rewrite of the sniffer in 3.16, and a;b;c;d,e;f;g;h is still detected as ,: the preferred delimiters break the tie between combinations which fit the sample equally well, and the number of occurrences does not matter. GH-154336 documents this.

Whether the preferred attribute itself should be documented or become a parameter is a separate question: it should probably be considered together with the quote and escape characters.