re.sub appears not to check count optional argument for integerness

[DannyYoo]

BPO	28905
Nosy	@bitdancer
Superseder	bpo-28082: re: convert re flags to (much friendlier) IntFlag constants

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-12-08.18:37:08.796>
created_at = <Date 2016-12-08.17:02:58.334>
labels = ['type-bug', 'library']
title = 're.sub appears not to check count optional argument for integerness'
updated_at = <Date 2016-12-08.18:37:08.794>
user = 'https://bugs.python.org/DannyYoo'

bugs.python.org fields:

activity = <Date 2016-12-08.18:37:08.794>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2016-12-08.18:37:08.796>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2016-12-08.17:02:58.334>
creator = 'Danny Yoo'
dependencies = []
files = []
hgrepos = []
issue_num = 28905
keywords = []
message_count = 4.0
messages = ['282719', '282721', '282722', '282726']
nosy_count = 2.0
nosy_names = ['r.david.murray', 'Danny Yoo']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '28082'
type = 'behavior'
url = 'https://bugs.python.org/issue28905'
versions = ['Python 3.4']

[DannyYoo]

This comes from diagnosing a beginner's question on Python-tutor.

https://mail.python.org/pipermail/tutor/2016-December/110066.html

It appears that re.sub is not checking whether the count argument is integer or not, and silently accepts a nonsensical argument. For example:

>>> import re
>>> s = "AAAcBBB\nAAAdBBB"
>>> print(re.sub(r'^AAA', "aaa", s, re.MULTILINE))
aaacBBB
AAAdBBB

Of course, the user intended to pass re.MULTILINE to flags, not to count, but the fact that this isn't raising a TypeError is error-prone.

[DannyYoo]

Ugh. I suddenly realize that this is complicated by the fact that flag values are themselves represented as integers, and Python's type system isn't rich enough to label flag values as a distinct type for the purposes.

It may be worthwhile to add a warning in the documentation about this, as it is an easy mistake to make.

[DannyYoo]

Alternatively, change the representation of flag values from integers to some class extension that supports the common bitwise operators.

As a very rough sketch:

>>> class FlagInt(int):
... def __or__(self, other):
... return FlagInt(int(self) | int(other))
... 
>>> f1 = FlagInt(1)
>>> f2 = FlagInt(2)
>>> f1 | f2
3
>>> isinstance(3, FlagInt)
False
>>> isinstance(f1 | f2, FlagInt)
True

That way, flag arguments can be determined at runtime to have derived from the proper flag values.

This kind of approach may have some backwards-incompatibility, unfortunately, since other folks have been hardcoding integers rather than use the flag constants. Other concerns might include serialization, in case someone tries to save a FlagInt somewhere and pull it out at some other time.

[bitdancer]

See bpo-28082.