Inconsistent error messages when returning the wrong type for the type-conversion magic methods

[mxgordon]

Bug report

Bug description:

I noticed the error messages between magic methods like __int__ and __float__ were inconsistent. This seems like slightly undesirable behavior to me. I used the following code to generate many of them.

class Foo:
 def __int__(self):
 return None
 
 def __float__(self):
 return None
 
 def __bytes__(self):
 return None
 
 def __complex__(self):
 return None
 
 def __bool__(self):
 return None
 
 def __str__(self):
 return None

try:
 int(Foo())
except Exception as e:
 print(e)

try:
 float(Foo())
except Exception as e:
 print(e)

try:
 bytes(Foo())
except Exception as e:
 print(e)

try:
 complex(Foo())
except Exception as e:
 print(e)

try:
 bool(Foo())
except Exception as e:
 print(e)

try:
 str(Foo())
except Exception as e:
 print(e)

And the output is as follows:

__int__ returned non-int (type NoneType)
Foo.__float__ returned non-float (type NoneType)
__bytes__ returned non-bytes (type NoneType)
__complex__ returned non-complex (type NoneType)
__bool__ should return bool, returned NoneType
__str__ returned non-string (type NoneType)

The first issue I've made, but seems like a reasonable bug. I'm not sure if there are other "type-conversion" magic methods out there that aren't consistent.

CPython versions tested on:

3.13, 3.12

Operating systems tested on:

Windows

Linked PRs

• gh-130821: Add type information to wrong type error messages #130835
• gh-130821: Make error messages consistent in Modules/ #144737
• gh-130821: Add type information to error messages for remaining magic methods #144827
• gh-130821: Add type info to error messages for __len__ and __sizeof__ #151894
• gh-130821: Improve error messages for magic methods returning wrong type #154606

[skirpichev]

The difference is

cpython/Objects/abstract.c

Lines 1529 to 1534 in d780f0a

	if (!PyLong_Check(result)) {
	PyErr_Format(PyExc_TypeError,
	"__int__ returned non-int (type %.200s)",
	Py_TYPE(result)->tp_name);
	Py_DECREF(result);
	return NULL;

vs

cpython/Objects/abstract.c

Lines 1607 to 1612 in d780f0a

	if (!PyFloat_Check(res)) {
	PyErr_Format(PyExc_TypeError,
	"%.50s.__float__ returned non-float (type %.50s)",
	Py_TYPE(o)->tp_name, Py_TYPE(res)->tp_name);
	Py_DECREF(res);
	return NULL;

CC @serhiy-storchaka as author of 16931c3.

I don't think that inconsistency is a real bug. But that does make sense for me as a feature request: i.e. include type information to the exception message.

[ZeroIntensity]

I've marked this as easy for anyone that wants to dive into the C code for the first time.

[donbarbos]

if no one minds I'm going to send a PR with a unified message using as a reference message: Foo.__float__ returned non-float (type NoneType)

[donbarbos]

but after "grep" sources i found more cases with "returned non-" template, excluding tests and docs:

grep output

./Objects/abstract.c: "__index__ returned non-int (type %.200s)",
./Objects/abstract.c: "__index__ returned non-int (type %.200s). "
./Objects/abstract.c: "__int__ returned non-int (type %.200s)",
./Objects/abstract.c: "__int__ returned non-int (type %.200s). "
./Objects/abstract.c: "%.50s.__float__ returned non-float (type %.50s)",
./Objects/abstract.c: "%.50s.__float__ returned non-float (type %.50s). "
./Objects/abstract.c: "iter() returned non-iterator "
./Objects/bytesobject.c: "__bytes__ returned non-bytes (type %.200s)",
./Objects/bytesobject.c: "__bytes__ returned non-bytes (type %.200s)",
./Objects/complexobject.c: "__complex__ returned non-complex (type %.200s)",
./Objects/complexobject.c: "__complex__ returned non-complex (type %.200s). "
./Objects/funcobject.c: PyErr_Format(PyExc_TypeError, "__annotate__ returned non-dict of type '%.100s'",
./Objects/moduleobject.c: PyErr_Format(PyExc_TypeError, "__annotate__ returned non-dict of type '%.100s'",
./Objects/object.c: "__repr__ returned non-string (type %.200s)",
./Objects/object.c: "__str__ returned non-string (type %.200s)",
./Objects/object.c: "__bytes__ returned non-bytes (type %.200s)",
./Objects/fileobject.c: "object.readline() returned non-string");
./Objects/floatobject.c: "%.50s.__float__ returned non-float (type %.50s)",
./Objects/floatobject.c: "%.50s.__float__ returned non-float (type %.50s). "
./Objects/genobject.c: "__await__() returned non-iterator "
./Objects/typeobject.c: PyErr_Format(PyExc_TypeError, "__annotate__ returned non-dict of type '%.100s'",
./Objects/typeobject.c: "__buffer__ returned non-memoryview object");
./Lib/subprocess.py: return "Command '%s' returned non-zero exit status %d." % (
./Modules/_datetimemodule.c: "divmod() returned non-tuple (type %.200s)",
./Modules/_abc.c: "items() returned non-iterable");
./Modules/_pickle.c: "read() returned non-bytes object (%R)",

should we add type information to other methods and to methods from modules in stdlib?

[serhiy-storchaka]

Looks like a nice idea to me.

You can use %T and %N to format type names in C.

[donbarbos]

but after "grep" sources i found more cases with "returned non-" template, excluding tests and docs:
grep output

should we add type information to other methods and to methods from modules in stdlib?

here are more examples of error messages:

__len__() should return >= 0
__hash__ method should return an integer
__init__() should return None, not '%.200s'
__sizeof__() should return >= 0
__getnewargs_ex__ should return a tuple, not '%.200s'
__getnewargs_ex__ should return a tuple of length 2, not %zd
__length_hint__() should return >= 0
__length_hint__ must be an integer, not %.100s
read() should return bytes
readall() should return bytes
decoder should return a string result, not '%.200s'
encoder should return a bytes object, not '%.200s'
iterator should return strings, not %.200s (the file should be opened in text mode)

I suggest using the following template for all of them: {TYPE}.__method__ should return ..., not ... (or must return)
unfortunately we don't have a PEP about error messages, but we had a similar PR #127345 and I found discuss

[serhiy-storchaka]

I like it. But since it's such a wide change, we need approval from more core developers. Could you open a discussion at https://discuss.python.org/?

I think also that there should be parentheses after the method name. This would help to distinguish them from attributes.