False positive when mixing Mapping and Union

[wkschwartz]

A function that expects a heterogeneous dictionary and gets a dictionary with a subset of the expected types seems to cause problems.

from typing import Any, Mapping, Union
def f(arg: Mapping[ str , Any]) -> None: pass
def g(arg: Mapping[Union[str, int], Any]) -> None: pass
f({'b': 'c'})
g({'b': 'c'})
d = {'b': 'c'}
f(d)
g(d) # <--- Line 8

Mypy's output is

mypy-test.py:8: error: Argument 1 to "g" has incompatible type "Dict[str, str]"; expected "Mapping[Union[str, int], Any]"

I'm using Python 3.7.1 and Mypy 0.641. Possibly related to #5849?

[JelleZijlstra]

This is because Mapping is invariant in its key type. See https://mypy.readthedocs.io/en/latest/common_issues.html#invariance-vs-covariance.

[wkschwartz]

I'm having a hard time understanding how this isn't a bug:

1. str should satisfy Union[str, int]: str is not a subclass of Union[str, int] so variance shouldn't matter.
2. g({'b': 'c'}) should be equivalent to d = {'b': 'c'}; g(d).
3. The docs you linked says using immutable types mitigates the unexpected behavior—Mapping is an immutable type.

[ilevkivskyi]

Mutability is not a root cause, the root cause is invariance. Mutable collections are typically invariant, but an immutable collection can be invariant as well.

[JelleZijlstra]

python/typing#273 discusses why Mapping's key type is invariant.

[wkschwartz]

What about items 1 and 2? How can invariance prevent str from satisfying Union[str, int]? Why would it affect variables but not literals?

[JelleZijlstra]

2. is due to type context, where mypy infers a different type if you're accessing something in a context where a particular type is expected.

3. My understanding is that str is a subtype (though not subclass) of Union[str, int].

[bluenote10]

FYI related discussion python/typing#445 and feature request python/typing_extensions#5

[randolf-scholz]

This is not type safe for the following reason:

• g may be using the arg.__getitem__ function.
• Imagine, we had a UserMapping class that is a Mapping[UserString, Any] type that used UserString keys, a subclass of str with an additional method transliterate. Furthermore, assume UserMapping made use of that UserString.transliterate method in the UserMapping.__getitem__ call. (Imagine for example that the Mapping transliterates Greek to Latin characters before attempting a lookup.)
• Then, if we pass an instance of this UserMapping to g, g might attempt to perform a lookup with a regular string that lacks the .transliterate method. Thus it is not type safe.

In essence, because Mapping has both methods where the keys appear as input and output, Mapping needs to be invariant in the keys.

---

A way around this is:

from typing import Any, Mapping, Union, TypeVar

StrVar = TypeVar("StrVar", bound=str)
StrOrIntVar = TypeVar("StrOrIntVar", bound=Union[str, int])

def f(arg: Mapping[ StrVar , Any]) -> None: pass
def g(arg: Mapping[StrOrIntVar, Any]) -> None: pass
f({'b': 'c'})
g({'b': 'c'})
d = {'b': 'c'}
f(d)
g(d)

Using a TypeVar kind of acts as an @overload for every possible str class.

---

Of course, in the particular case of str as a subtype of Union[str, int] it would be nice if the type-checker were a bit smarter. Because here really str is kind of stronger than just a subtype.