Code readability: rename InterpreterFrame to `_Py_framedata`

[ncoghlan]

BPO	44800
Nosy	@ncoghlan, @vstinner, @markshannon, @pablogsal, @brandtbucher
PRs	WIP bpo-44800: Rename _PyInterpreterFrame to _Py_framedata #27525 [PEP 558 - WIP] bpo-30744: Trace hooks no longer reset closure state #3640 bpo-44800: rename _PyInterpreterFrame to _Py_frame #31987 bpo-44800: unobtrusively rename _PyInterpreterFrame to _Py_frame #32024 bpo-44800: Document internal frame naming conventions #32281 Revert "bpo-44800: Document internal frame naming conventions" #32301
Dependencies	bpo-44590: Create frame objects lazily when needed

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = None
created_at = <Date 2021-08-01.01:48:31.010>
labels = ['type-feature', '3.11']
title = 'Code readability: rename InterpreterFrame to `_Py_framedata`'
updated_at = <Date 2022-04-04.14:09:53.084>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2022-04-04.14:09:53.084>
actor = 'Mark.Shannon'
assignee = 'ncoghlan'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-08-01.01:48:31.010>
creator = 'ncoghlan'
dependencies = ['44590']
files = []
hgrepos = []
issue_num = 44800
keywords = ['patch']
message_count = 10.0
messages = ['398675', '398681', '398690', '399381', '399581', '399582', '414446', '416610', '416613', '416671']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'vstinner', 'Mark.Shannon', 'pablogsal', 'brandtbucher']
pr_nums = ['27525', '3640', '31987', '32024', '32281', '32301']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue44800'
versions = ['Python 3.11']

[ncoghlan]

When merging the bpo-44590 changes into my PEP-558 implementation branch, I found it very hard to follow when the code was referring to the new interpreter frames rather than the existing Python frame objects that were historically used for both execution and introspection.

The "interpreter frame" name was also a little confusing, since the introspection frames are still associated with a specific interpreter, they're just not required for code execution anymore, only for code introspection APIs that call for a full Python object.

So, inspired by the "gi_xframe" (etc) attributes added in #27077, I'm proposing the following internal refactoring:

• Rename "pycore_frame.h" to "pycore_xframe.h"
• Rename the _interpreter_frame struct to _execution_frame
• Rename the type from InterpreterFrame to ExecFrame
• Use "xf_" rather than "f_" as the struct field prefix on execution frames
• Use "xframe" and "xf" rather than "frame" and "f" for execution frame variables
• Consistently use _PyExecFrame as the access function prefix, rather than a confusing mixture of _PyFrame and _PyInterpreterFrame
• Rename _PyThreadState_PushFrame to _PyThreadState_PushExecFrame
• Rename _PyThreadState_PopFrame to _PyThreadState_PopExecFrame

[ncoghlan]

As a side effect of working on this, I noticed some changes that are needed to adapt the GDB integration hooks to the new frame state layout.

[ncoghlan]

PR for this proposed refactoring is now up, with a review requested from Mark: #27525

The PR mostly follows what I originally posted, except that I went with _Py_execution_frame and _PyExecFrame for the struct and typedef names, since these are present in a public header.

The outdated GDB hooks I found were in the gdbinit example file, rather than the actual libpython.py file that test_gdb covers. I suspect the gdbinit example will need further adjustments to actually work with Python 3.11, so rather than fully updating the implementation dependent pieces, I just added a comment suggesting it be checked after the interpreter optimisation development for 3.11 settles down.

[ncoghlan]

Mark raised some valid concerns with the proposed naming convention over on the PR:

• the proposed names make it sound like there are genuinely two kinds of frame, when the actual relationship is between a frame's data storage and a Python object providing an interface to that storage
• _PyExecFrame looks like an actual Python type name, so we probably want something more like "Py_buffer" (where the lowercase name is intended to indicate that the struct isn't a full Python type)

Mark offered "activation record" as a possible name, but I'm going to see how "_Py_framedata" looks first (with "fdata" as the abbreviated form, since "fd" is so frequently used to mean "file descriptor")

I'm also going to see how the PR looks if both the frame and frame data struct keep their existing field prefixes - it may be that changing variable names will be enough to avoid ambiguity, in which case leaving the field names alone genuinely reduces code churn.

[ncoghlan]

PR has been updated with a new API proposal prompted by Mark's review comments on the original proposal.

• Rename "pycore_frame.h" to "pycore_framedata.h"
• Rename the _interpreter_frame struct to _Py_execution_frame
• Rename the type from InterpreterFrame to _Py_framedata
• Rather than 6 fields with no prefix, and 6 fields with the "f_" prefix, _Py_framedata fields now consistently have no prefix (globals, builtins, locals, code, lasti, and stack are affected by this)
• Use "fdata" rather than a mixture of "frame" and "f" for frame data variables
• Generators and coroutines use gi_fdata (etc) rather than gi_xframe as their field name
• Frame objects use f_fdata rather than f_frame as their field name
• Thread states use fdata rather than frame as their field name
• Consistently use _Py_frameobject as the access function prefix, rather than a confusing mixture of _PyFrame and _PyInterpreterFrame
• Leave the name of _PyThreadState_PushFrame alone
• Leave the name of _PyThreadState_PopFrame alone

[ncoghlan]

From a naming convention perspective, the code comments and NEWS entry in the PR now refer to "full frame objects" (PyFrameObject) and "frame data storage structs" (_Py_framedata) to avoid giving the misleading impression that introspection and execution frames are different Python types.

[vstinner]

Oh. I didn't know this issue. I recently made changes around PyFrameObject:

• Move PyFrameObject to the internal C API (see bpo-46836 for the rationale)
• Rename CFrame to _PyCFrame
• Rename InterpreterFrame to _PyInterpreterFrame

I prepared PRs for Cython, greenlet and gevent to use the internal C API pycore_frame.h to get the PyFrameObject structure:

https://bugs.python.org/issue46836#msg414283

For the short term, these projects should use the internal C API. But I sent a call to add getter and setter functions:

https://mail.python.org/archives/list/capi-sig@python.org/thread/RCT4SB5LY5UPRRRALEOHWEQHIXFNTHYF/

If possible, it would be great to have a public C API so these projects don't use the internal C API at all, that's being discussed at:

• C API for the frame stack faster-cpython/ideas#309
• https://bugs.python.org/issue40421

In terms of backward compatibility, since PyFrameObject is now part of the internal C API, we can break things. In practice... it's better to not break 3rd party code too often. See for example Brandt Bucher who is directly impacted by these changes:

https://bugs.python.org/issue46836#msg414279

[ncoghlan]

Core dev forum thread: https://discuss.python.org/t/proposal-rename-pyinterpreterframe-struct-as-py-framedata/14213

The conclusion from the forum thread and associated PRs was that any of the further renaming proposed didn't provide sufficient additional clarity to justify further code churn in a semi public API.

As a result, the final PR just documents the status quo in the internal C API frame header (since the conventions arising from the code-churn-minimising migration to a split data structure is hard to infer just from reading the code): #32281

[ncoghlan]

New changeset 124227c by Nick Coghlan in branch 'main':
bpo-44800: Document internal frame naming conventions (GH-32281)
124227c

[markshannon]

New changeset 8a349eb by Mark Shannon in branch 'main':
Revert "bpo-44800: Document internal frame naming conventions (GH-32281)" (bpo-32301)
8a349eb