Lazy loading conflicts with modules overriding ModuleType

[effigies]

Bug report

Bug description:

Discovered while working on #117178.

PEP 726 notes that the existing way for a module to override default methods is:

# mod.py

import sys
from types import ModuleType

CONSTANT = 3.14

class ImmutableModule(ModuleType):
 def __setattr__(name, value):
 raise AttributeError('Read-only attribute!')

 def __delattr__(name):
 raise AttributeError('Read-only attribute!')

sys.modules[__name__].__class__ = ImmutableModule

If I want to load this module lazily, I can use this MRE (adapted from #117178):

import sys
import importlib
import importlib.util

def lazy_import(name):
 spec = importlib.util.find_spec(name)
 loader = importlib.util.LazyLoader(spec.loader)
 spec.loader = loader
 module = importlib.util.module_from_spec(spec)
 sys.modules[name] = module
 loader.exec_module(module)
 return module

# Eagerly load the module
eager_module = importlib.import_module(sys.argv[1])
print(f"Eagerly loaded: {type(eager_module)}")
del sys.modules[sys.argv[1]]

# Lazy-load the module...
lazy_module = lazy_import(sys.argv[1])
print(f"Lazy-loaded: {type(lazy_module)}")
# ... and then trigger load by listing its contents
print(dir(lazy_module)[:5])
print(f"Fully loaded: {type(lazy_module)}")

This currently (with the fix in #117179) produces:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
Traceback (most recent call last):
 File "/home/chris/Projects/cpython/repro.py", line 23, in <module>
 print(dir(lazy_module)[:5])
 ~~~^^^^^^^^^^^^^
 File "/home/chris/Projects/cpython/Lib/importlib/util.py", line 223, in __getattribute__
 self.__class__ = types.ModuleType
 ^^^^^^^^^^^^^^
 File "/home/chris/Projects/cpython/mod.py", line 8, in __setattr__
 raise AttributeError('Read-only attribute!')
AttributeError: Read-only attribute!. Did you mean: '__doc__'?

Making the following change:

- self.__class__ = types.ModuleType
+ object.__setattr__(self, '__class__', types.ModuleType)

Results in:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
['CONSTANT', 'ImmutableModule', 'ModuleType', '__builtins__', '__cached__']
Fully loaded: <class 'module'>

If instead we change to:

- self.__class__ = types.ModuleType
+ if isinstance(self, _LazyModule):
+ object.__setattr__(self, '__class__', types.ModuleType)

We get:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
['CONSTANT', 'ImmutableModule', 'ModuleType', '__builtins__', '__cached__']
Fully loaded: <class 'mod.ImmutableModule'>

I believe making this change is the best way to respect the intentions of the module writer.

I've tagged this as a bug because these things have not worked together, but it might be more of a feature for them to start to, as neither is exactly standard practice.

CPython versions tested on:

3.11, 3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-117182: Allow lazily loaded modules to modify their own __class__ #117185