Eval/exec and comprehension scopes unclear in documentation

[brunoloff]

BPO	43605
Nosy	@terryjreedy, @congma
PRs	bpo-43605: Improve the documentation to exec() and eval() #25039

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-03-23.17:26:04.893>
labels = ['type-bug', '3.10', 'docs']
title = 'Eval/exec and comprehension scopes unclear in documentation'
updated_at = <Date 2021-03-27.15:06:53.886>
user = 'https://bugs.python.org/brunoloff'

bugs.python.org fields:

activity = <Date 2021-03-27.15:06:53.886>
actor = 'congma'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2021-03-23.17:26:04.893>
creator = 'bruno.loff'
dependencies = []
files = []
hgrepos = []
issue_num = 43605
keywords = ['patch']
message_count = 9.0
messages = ['389397', '389469', '389565', '389569', '389592', '389593', '389595', '389597', '389599']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'docs@python', 'congma', 'bruno.loff']
pr_nums = ['25039']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43605'
versions = ['Python 3.10']

Linked PRs

• gh-87771: Improve documentation of eval() #100003

[brunoloff]

Python 3.9.2 seems to be giving me some unexpected difficulty evaluating generators inside evals. Here is the example:

def func(l):
 def get(i):
 return l[i]


 print(sum(get(i) for i in range(len(l)))) # works as expected, prints 10
 print(eval("get(0) + get(1) + get(2) + get(3)")) # works just fine, prints 10

 # if __globals is set to locals(), it still works, prints 10
 print(eval("sum(get(i) for i in range(len(l)))", locals()))

 # This will complain
 print(eval("sum(get(i) for i in range(len(l)))"))

func([1,2,3,4])

The last line gives the following error

Traceback (most recent call last):
 File "/something/test_eval.py", line 28, in <module>
 func([1,2,3,4])
 File "/something/test_eval.py", line 10, in func
 print(eval("sum(get(i) for i in range(len(l)))")) # this does not work... bug?
 File "<string>", line 1, in <module>
 File "<string>", line 1, in <genexpr>

NameError: name 'get' is not defined

Any kind of generator-based code wont work. The following lines would give the same an error:

print(eval("sum(get(i) for i in range(len(l)))"), globals(), locals())
print(eval("[get(i) for i in range(len(l))]"))
print(eval("{i:get(i) for i in range(len(l))}"))

Any clue what is happening? The documentation on eval seems to give no insight on why this behavior is as is. This really feels like an issue, at the very least, it's an issue in the documentation.

[congma]

I think this is in the same class of behaviours as

def func(l):
 def get(i):
 return l[i]
 print(eval("(lambda x: get(x))(0)")) # Call anonymous lambda with the constant 0 as argument

Calls like func(["spam"]) will not "work", and NameError is raised.

In this case, inside the lambda's body the name "get" can't be resolved. For the lambda body, the name "get" is a nonlocal but there's no way to access a nonlocal in a lambda.

The comprehensions, like lambdas, are in their own nested scope.

[terryjreedy]

This is not an execution bug.

https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries

"However, aside from the iterable expression in the leftmost for clause, the comprehension is executed in a separate implicitly nested scope. This ensures that names assigned to in the target list don’t “leak” into the enclosing scope."

So when the passed in locals is not the passed in globals, that means a separate local scope.

https://docs.python.org/3/reference/expressions.html#generator-expressions is a little less clear.

"Variables used in the generator expression are evaluated lazily when the __next__() method is called for the generator object (in the same fashion as normal generators). However, the iterable expression in the leftmost for clause is immediately evaluated, so that an error produced by it will be emitted at the point where the generator expression is defined, rather than at the point where the first value is retrieved. Subsequent for clauses and any filter condition in the leftmost for clause cannot be evaluated in the enclosing scope as they may depend on the values obtained from the leftmost iterable."

By implication, the value expressions are also not evaluated in the enclosing local scope.

I am thinking about adding something to the eval/exec doc, but this issue overlaps with another one about their doc.

https://docs.python.org/3/reference/expressions.html#generator-expressions