Adressing the `object.__eq__` dilemma.

[randolf-scholz]

The Dilemma

# current annotation in typeshed
class object:
 def __eq__(self, value: object, /) -> bool: ...

This is too strict, because everything subclasses object and some classes may to wish to:

1. Narrow the range of allowed objects.
2. Use a return type other than bool (e.g. numpy array element-wise comparison, symbolic computation, etc.)

On the other hand, many APIs (incorrectly) assume that a == b always return a bool, so changing the return type away from bool can lead to problems.

Solving the Dilemma

Satisfying (1) is easy enough by changing value: object to value: Any. (one could maybe explore using Never also)

Satisfying (2) can be achieved in several ways, specifically I tested: (a) -> object, (b) -> Any and (c) -> Any | bool

• Option (a) leads to very bad primer result as per the dilemma explained above.

• Option (b) triggers a lot of no-any-return warnings with mypy when a function does somethign like return a==b.

• Option (c) is essentially a cheat code that avoids the no-any-return warnings from option (b)

In particular, for (c) the primer actually looks really good. We have ~50 fixed Unused "type: ignore" comment errors, and 16 suprious [override] errors eliminated, without any new errors.

---

References

• https://discuss.python.org/t/make-type-hints-for-eq-of-primitives-less-strict/34240
• https://discuss.python.org/t/method-eq-overrides-class-object-in-an-incompatible-manner-well-not-always/42878
• https://discuss.python.org/t/hash-eq-and-lsp/68138

Appendix: How python's a==b works internally

This cannot be properly annotated with the current type system, so all type checkers need to special case it.

def eq(left, right):
 left_type = type(left)
 right_type = type(right)

 # if RHS is proper subtype of LHS, try RHS's equality method
 if left_type != right_type and issubclass(right_type, left_type):
 result = right.__eq__(left)
 if result is not NotImplemented:
 return result

 # try LHS's equality method
 result = left.__eq__(right)
 if result is not NotImplemented:
 return result

 # try RHS's equality method
 result = right.__eq__(left)
 if result is not NotImplemented:
 return result
 
 return False # fallback if all methods return NotImplemented

[srittau]

Previous (partial) discussion: #3685. I still believe that implementations that intentionally violate the contract of __eq__ (which makes sense when building a DSL) should explicitly use # type: ignore.

[AlexWaygood]

I agree, I think it's an important soundness check, and I wouldn't want to hinder type checkers' ability to provide that soundness check. While there are some established libraries that do this, and it might make sense in certain very narrow situations in new code, I think these are the exceptions rather than the rule. I think it's overall very desirable for type checkers to complain about __eq__ methods that accept a narrower type than object or return a non-bool. That's usually a mistake that will have bad consequences and mean that type checkers are unable to spot bugs in your code elsewhere.

[randolf-scholz]

What about at least allowing Any for the argument type, and leaving the return type as-is?

As far as I understand it, type-checkers have more or less settled for interpreting the argument annotation of __eq__ as the types for which the method will not return NotImplemented.

So for instance if my class only supports __eq__ with Self | float and otherwise would return NotImplemented, I'd like to annotate it as such, but this will trigger override, even if th return is bool.

[AlexWaygood]

What about at least allowing Any for the argument type, and leaving the return type as-is?

But I think having an incorrect argument type is usually the way that most people write unsound __eq__ methods. If we made this change, type checkers wouldn't emit any errors on this snippet, even though it fails at runtime:

class Foo:
 x: int = 0

 def __eq__(self, other: Foo) -> bool:
 return self.x == other.x

def compare_objects(a: object, b: object):
 return a == b

compare_objects(Foo(), object())

[randolf-scholz]

The last example by @AlexWaygood is instructive, this makes me think that there's nothing that typeshed currently can do about the situation.

However, I do want to point out that many DSLs, like for example polars.Series already break this.
Moreover, I think it is likely that people who make this mistake in the first place will likely either:

1. use a bad type: igore[override]

2. break the design of operators by doing

    def __eq__(self, other: object, /) -> bool:
    return isinstance(other, Foo) and self.x == self.other

   rather than

   def __eq__(self, other: object, /) -> bool:
    if isinstance(other, Foo):
    return self.x == self.other
    return NotImplemented