gh-130999: Avoid exiting the new REPL when there are non-string candidates for suggestions

[devdanzin]

The new REPL will exit when given code like:

>>>importrunpy
... runpy._run_module_code("blech", {0: "", "bluch": ""}, "")
...

(See full traceback in #130999 (comment))

With this PR, the repr of the error causing the exit will be displayed instead:

>>>importrunpy
... runpy._run_module_code("blech", {0: "", "bluch": ""}, "")
...
TypeError("all elements in 'candidates' must be strings")
>>>

This is an easy, but perhaps too rough, solution. Suggestions for a better way to display the error (or even catch the exception) are very welcome.

• Issue: New REPL exits when there are non-string candidates for suggestions #130999

[pablogsal]

I am not convincing this is the right solution. This will hide the real error and as the traceback is missing will make it very difficult to find out what the problem is. The real solution is probably to filter the candidates in the traceback module before submitting it to the _generate_suggestions function.

@ambv@iritkatriel what do you think?

[iritkatriel]

I agree with @pablogsal.

Either filter in traceback, or error out in runpy when the globals dict is weird.

[devdanzin]

Thank you both for the review, sorry for taking so long to address it.

Either filter in traceback, or error out in runpy when the globals dict is weird.

runpy is just an easy path to a NameError with a user provided globals, but it also works with e.g.:

>>>g= {0:1, "p": 1}
>>>exec(compile("pa", 'g', "exec"), g)
[...]
NameError: name'pa'isnotdefinedDuringhandlingoftheaboveexception, anotherexceptionoccurred:
[...]
TypeError: allelementsin'candidates'mustbestrings
[AndtheREPLexits]

So, the current situation in trackeback._compute_suggestion_error() is:

• ImportError: if the module has a non-string in its __dict__, no suggestion is offered and no error happens.
• AttributeError: object returns a non-string in its __dir__ (or has one in __dict__), no suggestion is offered and no error happens.
• NameError: if f_locals, f_globals or f_builtins has a non-string key, an error is raised that exits the new REPL.

I suggest we just guard against an exception in the NameError case and offer no suggestions (by returning None). An alternative that gets the same result is to return None if a check like any(not isinstance(x, str) for x in candidates) succeeds.

Or we can filter non-strings from the candidates in the NameError case, which would provide suggestions while avoiding the error. If this is preferred, I suggest to also filter non-strings in the ImportError and AttributeError cases, for the same effect.

[tomasr8]

@devdanzin If you just want to fix the crash, this one-liner does the job:

diff --git a/Lib/traceback.py b/Lib/traceback.py
index 647c23ed782..cd5e2a88f07 100644
--- a/Lib/traceback.py+++ b/Lib/traceback.py@@ -1527,6 +1527,7 @@ def _compute_suggestion_error(exc_value, tb, wrong_name):
+ list(frame.f_globals)
+ list(frame.f_builtins)
)
+ d = [name for name in d if isinstance(name, str)]
# Check first if we are in a method and the instance
# has the wrong name as attribute

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[devdanzin]

Gentle ping @iritkatriel@pablogsal . Should we go with guarding against the exception and not offering suggestions on NameError case? What's your preference?

[terryjreedy]

Non-string keys are legal in dicts. Hints are an excellent by optional (and CPython-specific, I presume) enhancement. The hinters should just ignore, by filtering out, non-string keys and give a hint if possible. There is no reason to not give hint. Hinters are not linters.

[devdanzin]

A decision on this (#131001 (comment)) is blocking #130997. If no different opinions are presented, I'm going with @terryjreedy's approach in a day or two.

[ambv]

I agree with Terry and your final suggestion in the comment, let's filter and provide whatever other suggestions still apply. This is interactive, so unlikely to be "too slow for comfort".

[devdanzin]

I've implemented filtering of the candidates, and also guarded against dir() raising, so it's protected from exiting the REPL in more cases, while correctly displaying the remaining suggestions.

One doubt: when dir() raises, I chose to fetch obj.__dict__.keys(), but that misses attributes defined in the class, so I also set it to fetch from obj.__class__.__dict__.keys(). What is preferred, more candidates for suggestions (current code) or simpler/more strict suggestions (only fetch from the instance dict)?

[ambv]

@devdanzin What you did is what users will expect 👍🏻

[miss-islington-app]

Thanks @devdanzin for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-135019 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-135020 is a backport of this pull request to the 3.13 branch.

[ambv]

This PR causes failures when doing refleak hunting. I'm on it.