Allow variable redefinition if lifetimes don't overlap

[JukkaL]

As a follow-up to #1174 and #6197, we should perhaps allow two assignments to create independent variables whenever the lifetimes of the variables don't overlap (when using --allow-redefinitions).

Example:

if foo():
 x = 0
 print(x)
else:
 x = 'a' # Define a new variable?
 print(x)

Another example:

for x in [1, 2]:
 print(x)
if foo():
 for x in ['a', 'b']: # Define a new variable?
 print(x)

[ShadowLNC]

Not sure if this would need to be split into a separate issue, or if it is relevant here, but I have some code like this:

try:
 ...
except AssertionError as e:
 errors.append(e)

if condition:
 e = AssertionError("message")
 errors.append(e)

Essentially the same as Django's ValidationError pattern. mypy warns for the assignment to variable e outside the except block, and also the reading of the deleted variable, despite the fact that I've just assigned it.

[dumblob]

This is exactly what I see as a common pattern in code - quite often due to for loops or exceptions - exactly as in the above examples. So I'm also raising my hand for prioritizing this 😉.

[kshpytsya]

My case, which as far as I can tell is related to this issue, is with the following.

I am (ab)using context managers in the following pattern, which is used in some configuration building code written by end-users of my tool.

Heavily simplified:

toolmodule.py:

import contextlib
import typing

class A:
 a: int
 b: str

def get_A(name: str) -> typing.Generator[A, None, None]:
...

class B:
 c: bool

def get_B(name: str) -> typing.Generator[B, None, None]:
...

userwritten.py:

import toolmodule

with get_A("") as x:
 x.a = 1
 x.b = ""

with get_B("") as x:
 x.c = True

for i in [1]:
 with get_A("") as x: # <- error: Incompatible types in assignment (expression has type "A", variable has type "B")
 x.a = 1

(In actual code, get_X return special context managers returning accessor classes which fail in runtime once the scope of with block is exited.)

I suspected that there could be some nuances related to context managers, but the following code has the same problem:

import typing as tp


x = 1
if tp.TYPE_CHECKING:
 reveal_type(x)
print(x)


x = [1]
if tp.TYPE_CHECKING:
 reveal_type(x)
print(x)


if True:
 x = "" # <- error: Incompatible types in assignment (expression has type "str", variable has type "List[int]")
 if tp.TYPE_CHECKING:
 reveal_type(x)
 print(x)

for i in [1]:
 x = "" # <- error: Incompatible types in assignment (expression has type "str", variable has type "List[int]")
 if tp.TYPE_CHECKING:
 reveal_type(x)
 print(x)

Rhetorically and off-topic: I really wish Python had some scoping statement, akin to:

scoped:
 x = 1

x # <- error: undefined

scoped for i in []:
 ...

i # <- error: undefined

scoped with f() as y:
 ...

y # <- error: undefined

[DevilXD]

Is there any progress on this? Having type "redefinition" errors, just because I used a variable that's named the same as the one I used in a for loop couple of lines earlier, is getting quite tiresome. The flag that allows redefinitions doesn't sound that great for my usage case either.

IMO, if someone is using a variable in a for loop, it should redefine the type of the variable, even without the "allow redefinitions" flag - consider this:

my_list: List[int]
my_other_list: List[Thing]

for x in my_list:
 reveal_type(x) # int

reveal_type(x) # int (still, persistent)

for x in my_other_list: # redefines type of x
 reveal_type(x) # Thing

x = "test" # str (redefines type of x as well)

Perhaps the variable could be tagged as "reassignable" internally by MyPy somehow? This would also handle @ShadowLNC's case.

[arseniiv]

Can this be a related benign redefinition case too:

from pathlib import Path
for x in ['a/b']:
 reveal_type(x) # str, ok
 x = Path(x) # can’t redefine
 reveal_type(x) # str instead of Path
 reveal_type(x.stem) # error accessing, and Any because of that

With no for it works as expected, but in this manner it suddenly doesn’t (mypy 1.2, Python 3.11). I can’t see why a single iteration of a loop is different from an isolated block, as at the start of each iteration, x is reassigned a guaranteed str instance (if we know that what’s iterated over is Iterable[str])—but maybe there are pitfalls.

[filips123]

I'm not sure if this is the same issue, but I have a similar problem... I have the following code:

def get_menus(date: datetime.date) -> dict[str, Optional[dict[str, str]]]:
 # SnackMenu and LunchMenu are SQLAlchemy 2.0 models
 snack = Session.query(SnackMenu).filter(SnackMenu.date == date).first()
 lunch = Session.query(LunchMenu).filter(LunchMenu.date == date).first()

 reveal_type(snack) # Revealed type is "Union[gimvicurnik.database.SnackMenu, None]"
 reveal_type(lunch) # Revealed type is "Union[gimvicurnik.database.LunchMenu, None]"

 if snack:
 snack = {
 "normal": snack.normal,
 "poultry": snack.poultry,
 "vegetarian": snack.vegetarian,
 "fruitvegetable": snack.fruitvegetable,
 }

 if lunch:
 lunch = {
 "normal": lunch.normal,
 "vegetarian": lunch.vegetarian,
 }

 return {
 "snack": snack,
 "lunch": lunch,
 }

When running mypy with the following configuration (notice that allow_redefinition is enabled):

[tool.mypy]
python_version = 3.9
show_column_numbers = true
show_error_codes = true
allow_redefinition = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_configs = true
warn_unused_ignores = true
warn_no_return = true
warn_return_any = true
warn_unreachable = true

I get the following type errors:

gimvicurnik\blueprints\menus.py:29:25: error: Incompatible types in assignment (expression has type "Dict[str, Optional[str]]", variable has type "Optional[SnackMenu]") [assignment]
gimvicurnik\blueprints\menus.py:37:25: error: Incompatible types in assignment (expression has type "Dict[str, Optional[str]]", variable has type "Optional[LunchMenu]") [assignment]
gimvicurnik\blueprints\menus.py:43:17: error: Dict entry 0 has incompatible type "str": "Optional[SnackMenu]"; expected "str": "Optional[Dict[str, str]]" [dict-item]
gimvicurnik\blueprints\menus.py:44:17: error: Dict entry 1 has incompatible type "str": "Optional[LunchMenu]"; expected "str": "Optional[Dict[str, str]]" [dict-item]

I would expect that mypy would not reject this code, especially since allow_redefinition is enabled.