Skip to content

gh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit() - #97803

Merged
miss-islington merged 17 commits into
python:mainfrom
warsaw:issue-86298
Oct 7, 2022
Merged

gh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit()#97803
miss-islington merged 17 commits into
python:mainfrom
warsaw:issue-86298

Conversation

@warsaw

@warsawwarsaw commented Oct 3, 2022•
edited by miss-islington
Loading

Copy link
Copy Markdown
Member

In _warnings.c, in the C equivalent of warnings.warn_explicit(), if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning. To do this, it needs the module's loader.

Previously, it would only look up __loader__ in the module globals. In #86298 we want to defer to the __spec__.loader if available.

The first step on this journey is to check that loader == __spec__.loader and issue another warning if it is not. This commit does that.

Since this is a PoC, only manual testing for now.

# /tmp/foo.pyimportwarningsimportbarwarnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
# /tmp/bar.pyimportsysimportosimportpathlib# __loader__ = pathlib.Path()

Then running this: ./python.exe -Wdefault /tmp/foo.py

Produces:

bar.py:2: RuntimeWarning: warning!
import os

Uncomment the __loader__ = line in bar.py and try it again:

sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os

Automerge-Triggered-By: GH:warsaw

@warsaw
PoC for checking both __loader__ and __spec__.loader
37b581a
In _warnings.c, in the C equivalent of warnings.warn_explicit(), if the module
globals are given (and not None), the warning will attempt to get the source
line for the issued warning. To do this, it needs the module's loader.
Previously, it would only look up `__loader__` in the module globals. In
python#86298 we want to defer to the
`__spec__.loader` if available.
The first step on this journey is to check that `loader` == `__spec__.loader`
and issue another warning if it is not. This commit does that.
Since this is a PoC, only manual testing for now.
```python
import warnings
import bar
warnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
```
```python
import sys
import os
import pathlib
```
Then running this: `./python.exe -Wdefault /tmp/foo.py`
Produces:
```
bar.py:2: RuntimeWarning: warning!
import os
```
Uncomment the `__loader__ = ` line in `bar.py` and try it again:
```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os
```
@warsaw

warsaw commented Oct 3, 2022

Copy link
Copy Markdown
MemberAuthor

Tagging @brettcannon

@warsawwarsaw changed the title PoC for checking both __loader__ and __spec__.loaderissue-86298: PoC for checking both __loader__ and __spec__.loaderOct 3, 2022
@warsawwarsaw changed the title issue-86298: PoC for checking both __loader__ and __spec__.loader#86298: PoC for checking both __loader__ and __spec__.loaderOct 3, 2022
@warsawwarsaw changed the title #86298: PoC for checking both __loader__ and __spec__.loadergh-86298: PoC for checking both __loader__ and __spec__.loaderOct 3, 2022
@warsaw
warsaw marked this pull request as draft October 3, 2022 22:33
@warsawwarsaw self-assigned this Oct 3, 2022
@warsaw
warsaw marked this pull request as ready for review October 4, 2022 00:03
@warsaw

warsaw commented Oct 4, 2022

Copy link
Copy Markdown
MemberAuthor

Converting from draft since I now have tests.

@warsaw

warsaw commented Oct 4, 2022

Copy link
Copy Markdown
MemberAuthor

It's better to split these consistency fixes up into separate PRs, so this one only addresses the _warnings.c part of the issue.

@warsaw

warsaw commented Oct 4, 2022

Copy link
Copy Markdown
MemberAuthor

@brettcannon I think this branch is ready for review. It only changes the _warnings.c case, adds tests, and updates some documentation. We'll split any related changes outside of this file into separate PRs.

@warsawwarsaw mentioned this pull request Oct 4, 2022
Closed
@warsawwarsaw changed the title gh-86298: PoC for checking both __loader__ and __spec__.loadergh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit()Oct 4, 2022
@warsawwarsaw mentioned this pull request Oct 4, 2022
Open
brettcannon
brettcannon approved these changes Oct 5, 2022
View reviewed changes
Comment threadPython/_warnings.c Outdated
@warsaw
Use a helper function
1066b9f
@warsaw
Merge branch 'main' into issue-86298
4602366
@warsaw
Pass warnings tests
831f832
Note however that the semantics of the helper has changed, and can't be full
on loader blessing... yet.
@warsaw
Merge branch 'main' into issue-86298
37254cf
@warsaw

warsaw commented Oct 7, 2022

Copy link
Copy Markdown
MemberAuthor

@brettcannon Please take another look.

brettcannon
brettcannon approved these changes Oct 7, 2022
View reviewed changes
Comment threadLib/importlib/_bootstrap_external.py
Comment threadLib/importlib/_bootstrap_external.py Outdated
Comment threadLib/importlib/_bootstrap_external.py
Comment threadLib/importlib/_bootstrap_external.py
Comment threadLib/test/test_importlib/import_/test_helpers.py
Comment threadLib/test/test_importlib/import_/test_helpers.py
Comment threadPython/_warnings.c
warsawand others added 4 commits October 6, 2022 17:51
@warsaw@brettcannon
Update Lib/importlib/_bootstrap_external.py
5600c9c
Co-authored-by: Brett Cannon <brett@python.org>
@warsaw@brettcannon
Update Lib/test/test_importlib/import_/test_helpers.py
8631e53
Co-authored-by: Brett Cannon <brett@python.org>
@warsaw@brettcannon
Update Lib/test/test_importlib/import_/test_helpers.py
b1b0d86
Co-authored-by: Brett Cannon <brett@python.org>
@warsaw
Merge branch 'main' into issue-86298
823400c
@miss-islington

miss-islington commented Oct 7, 2022

Copy link
Copy Markdown
Contributor

Status check is done, and it's a success ✅.

@miss-islington

miss-islington commented Oct 7, 2022

Copy link
Copy Markdown
Contributor

Sorry, I can't merge this PR. Reason: Head branch was modified. Review and try the merge again..

@miss-islington

miss-islington commented Oct 7, 2022

Copy link
Copy Markdown
Contributor

Status check is done, and it's a success ✅.

@miss-islington
miss-islington merged commit 13d4489 into python:mainOct 7, 2022
@warsaw
warsaw deleted the issue-86298 branch October 7, 2022 02:50
carljm added a commit to carljm/cpython that referenced this pull request Oct 8, 2022
@carljm
Merge branch 'main' into typewatch
0de5aff
* main:
pythongh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit() (pythonGH-97803)
pythongh-82874: Convert remaining importlib format uses to f-str. (python#98005)
Docs: Fix backtick errors found by sphinx-lint (python#97998)
pythongh-97850: Remove deprecated functions from `importlib.utils` (python#97898)
Remove extra spaces in custom openSSL documentation. (python#93568)
pythonGH-90985: Revert "Deprecate passing a message into cancel()" (python#97999)
mpage pushed a commit to mpage/cpython that referenced this pull request Oct 11, 2022
@warsaw@mpage
pythongh-86298: Ensure that __loader__ and __spec__.loader agree in w…
2781fef
…arnings.warn_explicit() (pythonGH-97803)
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning. To do this, it needs the module's loader.
Previously, it would only look up `__loader__` in the module globals. In python#86298 we want to defer to the `__spec__.loader` if available.
The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not. This commit does that.
Since this is a PoC, only manual testing for now.
```python
# /tmp/foo.py
import warnings
import bar
warnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
```
```python
# /tmp/bar.py
import sys
import os
import pathlib
# __loader__ = pathlib.Path()
```
Then running this: `./python.exe -Wdefault /tmp/foo.py`
Produces:
```
bar.py:2: RuntimeWarning: warning!
import os
```
Uncomment the `__loader__ = ` line in `bar.py` and try it again:
```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os
```
Automerge-Triggered-By: GH:warsaw
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brettcannonbrettcannonbrettcannon approved these changes
@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

Assignees

@warsawwarsaw

Labels

topic-importlib

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@warsaw@miss-islington@brettcannon@bedevere-bot