Avoid raising OverflowError in len() when __len__() returns negative large value

[serhiy-storchaka]

BPO	29839
Nosy	@warsaw, @terryjreedy, @vstinner, @serhiy-storchaka, @orenmn
PRs	bpo-29839: Raise ValueError rather than OverflowError in len() for negative values. #701

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-04-16.07:05:36.437>
created_at = <Date 2017-03-17.19:52:14.485>
labels = ['interpreter-core', 'type-feature', '3.7']
title = 'Avoid raising OverflowError in len() when __len__() returns negative large value'
updated_at = <Date 2017-04-16.07:05:36.434>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-04-16.07:05:36.434>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-04-16.07:05:36.437>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2017-03-17.19:52:14.485>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 29839
keywords = []
message_count = 5.0
messages = ['289779', '289782', '290109', '291740', '291742']
nosy_count = 5.0
nosy_names = ['barry', 'terry.reedy', 'vstinner', 'serhiy.storchaka', 'Oren Milman']
pr_nums = ['701']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29839'
versions = ['Python 3.7']

[serhiy-storchaka]

For now len() raises ValueError if __len__() returns small negative integer and OverflowError if __len__() returns large negative integer.

>>> class NegativeLen:
... def __len__(self):
... return -10
... 
>>> len(NegativeLen())
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ValueError: __len__() should return >= 0
>>> class HugeNegativeLen:
... def __len__(self):
... return -sys.maxsize-10
... 
>>> len(HugeNegativeLen())
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer

Proposed patch makes it always raising ValueError.

[warsaw]

I was going to say that this is an API change, but given that without this, folks would have to catch both exceptions and now only have to catch one of them, it isn't.

[terryjreedy]

https://docs.python.org/3/library/functions.html#len * does not specify the exception. In such cases, we occasionally change exception in x.y.0 releases without prior notice other than News and What's New. Also, I think unnecessarily exposing a compile-switch dependent internal detail, as now, is almost a bug. So +1 to applying in 3.7

• The doc does not specify that 'length' cannot be non-negative. Perhaps it should, so no-one will think that they can hijack '__len__' to return something that is not a length, as usually understood.

[serhiy-storchaka]

New changeset baf9f29 by Serhiy Storchaka in branch 'master':
bpo-29839: Raise ValueError rather than OverflowError in len() for negative values. (#701)
baf9f29

[serhiy-storchaka]

The doc does not specify that 'length' cannot be non-negative.

It does. https://docs.python.org/3/reference/datamodel.html#object.\_\_len__

.. method:: object.__len__(self)

Called to implement the built-in function :func:`len`. Should return the length
of the object, an integer ``>=`` 0.