Narrowing str | dict[str, str] with isinstance and .get()

[Akuli]

Bug Report

To Reproduce

def first() -> None:
 tag_name: str | dict[str, str]
 if isinstance(tag_name, dict):
 tag_name = tag_name.get("foo", "bar")
 reveal_type(tag_name)

def second() -> None:
 tag_name: str | dict[str, str]
 if isinstance(tag_name, dict):
 tag_name = tag_name["foo"]
 reveal_type(tag_name)

Expected Behavior

str
str

Actual Behavior

Union[str, dict[str, str]]
str

Your Environment

latest mypy

[RedKnite5]

I've been looking at this and figured out so far that when mypy accepts the CallExpr node Union[builtins.str, builtins.dict[builtins.str, builtins.str]] is added to self.type_context in ExpressionChecker. This is propagated to apply_generic_arguments in applytype.py as context where it is used to deduce the return type of the call to get.

The problem I'm having is that I'm not familiar with how apply_generic_arguments works so I'm not sure where in the function it messes up and what possible solutions would actually totally break its intended function.

[Akuli]

I looked into this a bit too. Here's a more self-contained reproducer:

from __future__ import annotations
from typing import TypeVar

T = TypeVar("T")

class MyDict:
 def get_item(self) -> str: ...
 def get(self, x: T) -> str | T: ...

def first() -> None:
 tag_name: str | MyDict
 if isinstance(tag_name, MyDict):
 tag_name = tag_name.get("asd")
 reveal_type(tag_name)

def second() -> None:
 tag_name: str | MyDict
 if isinstance(tag_name, MyDict):
 tag_name = tag_name.get_item()
 reveal_type(tag_name)

Next I tried:

def third() -> None:
 tag_name: str | MyDict
 if isinstance(tag_name, MyDict):
 reveal_type(tag_name) # MyDict
 reveal_type(tag_name.get("asd")) # str
 tag_name = tag_name.get("asd")
 reveal_type(tag_name) # str | MyDict

When I do reveal_type(tag_name.get("asd")), that happens in a new type context, and works as expected. But when I do tag_name = tag_name.get("asd"), the type of tag_name is taken from the type context as a suggestion for what the type of tag_name.get("asd") might be. To make that happen, mypy resolves T to appropriately.

I think the bug is that str | MyDict is still in the type context by the time we get to the type-narrowed part of the code. When the type narrowing happens, the type of tag_name should become MyDict, and whatever gets str | MyDict into the type context is buggy: it's not looking up the type of the newly-narrowed local variable correctly.

[RedKnite5]

I think its about the assignment back to the typed expression. This also exhibits the same behavior and follows a similar data path, but doesn't have type narrowing in the same way.

from typing import TypeVar
T = TypeVar("T")

def func(default: T) -> str | T: ...

def example() -> None:
	tag_name: str | int
	tag_name = func("a")
	reveal_type(tag_name)

Its the union of a type and a generic that turns out to be the same type assigned back to a variable with the union type.
Or maybe I'm misunderstanding how the context stack is supposed to work.

If the type hint on tag_name is omitted:

def example() -> None:
	tag_name = func("a")
	reveal_type(tag_name)

then when it gets to the function infer_function_type_arguments_using_context it returns early without inferring anything and proceeds to get the right answer. So it could be that str | int shouldn't be in the context any more, but I'm not sure.

Edit: Made the example more minimal

[Akuli]

In my example the type of tag_name should be MyDict, not str | MyDict, because type narrowing has happened. While it would be nice for tag_name = func("a") to narrow to str, it's not what this issue is about: here the narrowing has happened already, but the type context isn't picking it up.

Your example, modified to match what I consider to be this issue's scope, would be:

from typing import TypeVar
T = TypeVar("T")

def func(default: T) -> str | T: ...

def example() -> None:
	tag_name: str | int
	if isinstance(tag_name, str):
		tag_name = func("a")
		reveal_type(tag_name) # str | int :(

[AlexWaygood]

Fixed in #14151