Skip to content

gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc()#141665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vstinner wants to merge 9 commits into from
Closed

gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc() #141665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
0e322e4
gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc()
vstinner Nov 17, 2025
757eada
Deprecate the old C API
vstinner Nov 17, 2025
4ceff72
Fix PyUnstable_FrameEvalFunction doc
vstinner Nov 17, 2025
0a8d7cd
Add tests
vstinner Nov 17, 2025
f694542
Update Modules/_testcapimodule.c
vstinner Nov 18, 2025
3e48d91
Fix copy/paste mistake
vstinner Nov 19, 2025
c167f44
Merge branch 'main' into eval_frame_func
vstinner Nov 19, 2025
c624f74
_PyFrameEvalFunction changelog
vstinner Nov 19, 2025
8fd495b
PyUnstable_InterpreterState_SetEvalFrameFunc() can now fail
vstinner Nov 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions Doc/c-api/init.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1438,34 +1438,43 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
.. versionadded:: 3.8


.. c:type:: PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag)
.. c:type:: PyObject* (*PyUnstable_FrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag)

Type of a frame evaluation function.

The *throwflag* parameter is used by the ``throw()`` method of generators:
if non-zero, handle the current exception.

.. versionadded:: 3.6
as ``_PyFrameEvalFunction``.

.. versionchanged:: 3.9
The function now takes a *tstate* parameter.

.. versionchanged:: 3.11
The *frame* parameter changed from ``PyFrameObject*`` to ``_PyInterpreterFrame*``.

.. c:function:: _PyFrameEvalFunction _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
.. versionchanged:: next
Renamed to :c:type:`!PyUnstable_FrameEvalFunction`. The old private name
is deprecated, but will be available until the API changes.

.. c:function:: PyUnstable_FrameEvalFunction PyUnstable_InterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)

Get the frame evaluation function.

See the :pep:`523` "Adding a frame evaluation API to CPython".

.. versionadded:: 3.9
.. versionadded:: next

.. c:function:: void _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, _PyFrameEvalFunction eval_frame)
.. c:function:: void PyUnstable_InterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, PyUnstable_FrameEvalFunction eval_frame)

Set the frame evaluation function.

Return ``0`` on success, or set an exception and return ``-1`` on error.

See the :pep:`523` "Adding a frame evaluation API to CPython".

.. versionadded:: 3.9
.. versionadded:: next


.. c:function:: PyObject* PyThreadState_GetDict()
Expand Down
7 changes: 7 additions & 0 deletions Doc/deprecations/c-api-pending-removal-in-3.20.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,3 +7,10 @@ Pending removal in Python 3.20
representation.

* Macros :c:macro:`!Py_MATH_PIl` and :c:macro:`!Py_MATH_El`.

* :c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` functions are deprecated
and will be removed in Python 3.20. Use
:c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc` and
:c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc` instead.
Comment thread
efimov-mikhail marked this conversation as resolved.
(Contributed by Victor Stinner in :gh:`141518`.)
2 changes: 1 addition & 1 deletion Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2305,7 +2305,7 @@ Porting to Python 3.11
be used for ``size``.
(Contributed by Kumar Aditya in :issue:`46608`.)

* :c:func:`_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*``
* :c:func:`!_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*``
as its second parameter, instead of ``PyFrameObject*``.
See :pep:`523` for more details of how to use this function pointer type.

Expand Down
15 changes: 15 additions & 0 deletions Doc/whatsnew/3.15.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1091,6 +1091,14 @@ New features
* Add :c:func:`PyTuple_FromArray` to create a :class:`tuple` from an array.
(Contributed by Victor Stinner in :gh:`111489`.)

* Add a PyUnstable API for :pep:`523`:

* :c:type:`PyUnstable_FrameEvalFunction` type
* :c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc`
* :c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc`

(Contributed by Victor Stinner in :gh:`141518`.)

* Add :c:func:`PyUnstable_Object_Dump` to dump an object to ``stderr``.
It should only be used for debugging.
(Contributed by Victor Stinner in :gh:`141070`.)
Expand DownExpand Up@@ -1227,6 +1235,13 @@ Deprecated C APIs
since 3.15 and will be removed in 3.20.
(Contributed by Sergey B Kirpichev in :gh:`141004`.)

* :c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` functions are deprecated
and will be removed in Python 3.20. Use
:c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc` and
:c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc` instead.
(Contributed by Victor Stinner in :gh:`141518`.)
Comment thread
efimov-mikhail marked this conversation as resolved.


.. Add C API deprecations above alphabetically, not here at the end.

Expand Down
24 changes: 20 additions & 4 deletions Include/cpython/pystate.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,10 +302,26 @@ PyAPI_FUNC(void) PyThreadState_DeleteCurrent(void);

/* Frame evaluation API */

typedef PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, struct _PyInterpreterFrame *, int);
typedef PyObject* (*PyUnstable_FrameEvalFunction)(PyThreadState *tstate, struct _PyInterpreterFrame *, int);

PyAPI_FUNC(_PyFrameEvalFunction) _PyInterpreterState_GetEvalFrameFunc(
PyAPI_FUNC(PyUnstable_FrameEvalFunction) PyUnstable_InterpreterState_GetEvalFrameFunc(
PyInterpreterState *interp);
PyAPI_FUNC(void) _PyInterpreterState_SetEvalFrameFunc(
PyAPI_FUNC(int) PyUnstable_InterpreterState_SetEvalFrameFunc(
PyInterpreterState *interp,
_PyFrameEvalFunction eval_frame);
PyUnstable_FrameEvalFunction eval_frame);

// Deprecated aliases kept for backward compatibility
#define _PyFrameEvalFunction PyUnstable_FrameEvalFunction

Py_DEPRECATED(3.15) static inline PyUnstable_FrameEvalFunction
_PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
{
return PyUnstable_InterpreterState_GetEvalFrameFunc(interp);
}

Py_DEPRECATED(3.15) static inline void
_PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
PyUnstable_FrameEvalFunction eval_frame)
{
PyUnstable_InterpreterState_SetEvalFrameFunc(interp, eval_frame);
}
4 changes: 2 additions & 2 deletions Include/internal/pycore_interp_structs.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -87,7 +87,7 @@ struct _ceval_runtime_state {
struct trampoline_api_st trampoline_api;
FILE *map_file;
Py_ssize_t persist_after_fork;
_PyFrameEvalFunction prev_eval_frame;
PyUnstable_FrameEvalFunction prev_eval_frame;
#else
int _not_used;
#endif
Expand DownExpand Up@@ -860,7 +860,7 @@ struct _is {
PyObject *sysdict_copy;
PyObject *builtins_copy;
// Initialized to _PyEval_EvalFrameDefault().
_PyFrameEvalFunction eval_frame;
PyUnstable_FrameEvalFunction eval_frame;

PyFunction_WatchCallback func_watchers[FUNC_MAX_WATCHERS];
// One bit is set for each non-NULL entry in func_watchers
Expand Down
6 changes: 3 additions & 3 deletions Misc/NEWS.d/3.9.0a5.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1305,6 +1305,6 @@ and undefined PY_SSIZE_T_CLEAN whwn an exception is set.
.. section: C API

Add a private API to get and set the frame evaluation function: add
:c:func:`_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`_PyInterpreterState_SetEvalFrameFunc` C functions. The
:c:type:`_PyFrameEvalFunction` function type now takes a *tstate* parameter.
:c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` C functions. The
:c:type:`!_PyFrameEvalFunction` function type now takes a *tstate* parameter.
7 changes: 7 additions & 0 deletions Misc/NEWS.d/next/C_API/2025-11-17-16-08-11.gh-issue-141518.Z8ig9U.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
Add a PyUnstable API for :pep:`523`:

* :c:type:`PyUnstable_FrameEvalFunction` type
* :c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc`
* :c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc`

Patch by Victor Stinner.
28 changes: 28 additions & 0 deletions Modules/_testcapimodule.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2595,6 +2595,32 @@ create_managed_weakref_nogc_type(PyObject *self, PyObject *Py_UNUSED(args))
}


static PyObject *
noop_eval(PyThreadState *tstate, struct _PyInterpreterFrame *f, int exc)

@encukouencukouNov 19, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't _PyInterpreterFrame, need a public name too? Otherwise you can't quite use the functions without private API.

@vstinnervstinnerNov 19, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To implement an eval function, you should likely use the internal C API (pycore_interpframe.h) to access frame members, you're right. I don't want to add _PyInterpreterFrame structure members to the PyUnstable API, it's too unstable :-D I prefer to leave it in the internal C API.

I don't think that it's worth it to expose struct _PyInterpreterFrame*type in the PyUnstable API. What do you think?

@encukouencukouNov 19, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are rather useless without making _PyInterpreterFrame public. At least should be an opaque type, with a function to upgrade it to the full PyFrameObject.

@vstinnervstinnerNov 19, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API changed in Python 3.11 to take a _PyInterpreterFrame* instead of a PyFrameObject*. It's basically only used by PyTorch Dynamo which already uses the internal C API:

https://github.com/pytorch/pytorch/blob/be33b7faf685560bb618561b44b751713a660337/torch/csrc/dynamo/eval_frame.c#L15-L17

I wouldn't say that this API is useless without a public API for _PyInterpreterFrame*. It's just "unusual" :-)

At least should be an opaque type, with a function to upgrade it to the full PyFrameObject.

It would make the code (way) slower, _PyInterpreterFrame* idea is to avoid creating a concrete Python object for a frame unless it's strictly needed. I suggest using the internal C API for _PyInterpreterFrame* instead.

@encukouencukouNov 19, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say that this API is useless without a public API for _PyInterpreterFrame*.

What is the use, then?

@vstinnervstinnerNov 20, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emmatyping@efimov-mikhail: Do you have an opinion on these questions?

@efimov-mikhailefimov-mikhailNov 20, 2025•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, the main idea of this change is not really to make those functions public. We mark them as PyUnstable_ for sending clear message to its users: "we remember about those function, we don't want to change or remove them in the near future". One can use them and rely on their presence in C API, but can't rely on stability of _PyInterpreterFrame structure. So, there's a need to use some private headers in this case.

I agree that it's not the perfect position for us, but current situation isn't any better.
We have functions that both private and public.
They're private, because they have _Py* names.
But they're "kinda public", since PEP 523 is guaranteed their presence.
PyUnstable_ suits better for those semi-public functions.
But making _PyInterpreterFrame and all its members PyUnstable_ too seems redundant to me.

@encukouencukouNov 24, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But making _PyInterpreterFrame and all its members PyUnstable_ too seems redundant to me.

Same here. I think it should be opaque, and we should add (slow) a function to convert it to the full public PyFrameObject*.

That would make this usable without resorting to private API. Yes, doing that would be slow for the current users, so those will probably continue using the fully-private API -- but that's not the point.
If we expose PyUnstable_InterpreterState_GetEvalFrameFunc,

  • users should be able to use it (simple slow printf-based debugging is a valid use case), and
  • in CPython we should be able to test that you can use it without needing internal headers.

Or for a different point of view: really the issue solved here is that we don't want documented _Py-prefixed API.
That should apply to argument types too.

@vstinnervstinnerNov 25, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created #141950 to add PyUnstable_InterpreterFrame_GetFrameObject().

{
return NULL;
}

static PyObject *
test_interpreter_setevalframefunc(PyObject *self, PyObject *Py_UNUSED(args))
{
PyInterpreterState *interp = PyInterpreterState_Get();
PyUnstable_FrameEvalFunction eval_func;
int res;

eval_func = PyUnstable_InterpreterState_GetEvalFrameFunc(interp);

res = PyUnstable_InterpreterState_SetEvalFrameFunc(interp, noop_eval);
assert(res == 0);
assert(PyUnstable_InterpreterState_GetEvalFrameFunc(interp) == noop_eval);

res = PyUnstable_InterpreterState_SetEvalFrameFunc(interp, eval_func);
assert(res == 0);

Py_RETURN_NONE;
}


static PyMethodDef TestMethods[] = {
{"set_errno", set_errno, METH_VARARGS},
{"test_config", test_config, METH_NOARGS},
Expand DownExpand Up@@ -2691,6 +2717,8 @@ static PyMethodDef TestMethods[] = {
{"toggle_reftrace_printer", toggle_reftrace_printer, METH_O},
{"create_managed_weakref_nogc_type",
create_managed_weakref_nogc_type, METH_NOARGS},
{"test_interpreter_setevalframefunc",
Comment thread
efimov-mikhail marked this conversation as resolved.
test_interpreter_setevalframefunc, METH_NOARGS},
{NULL, NULL} /* sentinel */
};

Expand Down
11 changes: 9 additions & 2 deletions Modules/_testinternalcapi.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -656,8 +656,12 @@ decode_locale_ex(PyObject *self, PyObject *args)
static PyObject *
set_eval_frame_default(PyObject *self, PyObject *Py_UNUSED(args))
{
PyInterpreterState *interp = _PyInterpreterState_GET();
PyUnstable_FrameEvalFunction eval_func = _PyEval_EvalFrameDefault;
if (PyUnstable_InterpreterState_SetEvalFrameFunc(interp, eval_func) < 0) {
return NULL;
}
module_state *state = get_module_state(self);
_PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), _PyEval_EvalFrameDefault);
Py_CLEAR(state->record_list);
Py_RETURN_NONE;
}
Expand DownExpand Up@@ -689,7 +693,10 @@ set_eval_frame_record(PyObject *self, PyObject *list)
return NULL;
}
Py_XSETREF(state->record_list, Py_NewRef(list));
_PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), record_eval);
PyInterpreterState *interp = _PyInterpreterState_GET();
if (PyUnstable_InterpreterState_SetEvalFrameFunc(interp, record_eval) < 0) {
return NULL;
}
Py_RETURN_NONE;
}

Expand Down
28 changes: 21 additions & 7 deletions Python/perf_trampoline.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -470,7 +470,9 @@ _PyPerfTrampoline_SetCallbacks(_PyPerf_Callbacks *callbacks)
}
#ifdef PY_HAVE_PERF_TRAMPOLINE
if (trampoline_api.state) {
_PyPerfTrampoline_Fini();
if (_PyPerfTrampoline_Fini() < 0) {
return -1;
}
}
trampoline_api.init_state = callbacks->init_state;
trampoline_api.write_state = callbacks->write_state;
Expand All@@ -486,12 +488,18 @@ _PyPerfTrampoline_Init(int activate)
#ifdef PY_HAVE_PERF_TRAMPOLINE
PyThreadState *tstate = _PyThreadState_GET();
if (!activate) {
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, prev_eval_frame);
if (PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp,
prev_eval_frame) < 0) {
return -1;
}
perf_status = PERF_STATUS_NO_INIT;
}
else if (tstate->interp->eval_frame != py_trampoline_evaluator) {
prev_eval_frame = _PyInterpreterState_GetEvalFrameFunc(tstate->interp);
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, py_trampoline_evaluator);
prev_eval_frame = PyUnstable_InterpreterState_GetEvalFrameFunc(tstate->interp);
if (PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp,
py_trampoline_evaluator) < 0) {
return -1;
}
extra_code_index = _PyEval_RequestCodeExtraIndex(NULL);
if (extra_code_index == -1) {
return -1;
Expand All@@ -517,7 +525,9 @@ _PyPerfTrampoline_Fini(void)
}
PyThreadState *tstate = _PyThreadState_GET();
if (tstate->interp->eval_frame == py_trampoline_evaluator) {
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, NULL);
if (PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp, NULL) < 0) {
return -1;
}
}
if (perf_status == PERF_STATUS_OK) {
trampoline_api.free_state(trampoline_api.state);
Expand DownExpand Up@@ -553,7 +563,9 @@ _PyPerfTrampoline_AfterFork_Child(void)
if (perf_trampoline_type != PERF_TRAMPOLINE_TYPE_MAP) {
return PyStatus_Error("Failed to copy perf map file as perf trampoline type is not type map.");
}
_PyPerfTrampoline_Fini();
if (_PyPerfTrampoline_Fini() < 0) {
return PyStatus_Error("_PyPerfTrampoline_Fini() failed");
}
char filename[256];
pid_t parent_pid = getppid();
snprintf(filename, sizeof(filename), "/tmp/perf-%d.map", parent_pid);
Expand All@@ -563,7 +575,9 @@ _PyPerfTrampoline_AfterFork_Child(void)
} else {
// Restart trampoline in file in child.
int was_active = _PyIsPerfTrampolineActive();
_PyPerfTrampoline_Fini();
if (_PyPerfTrampoline_Fini() < 0) {
return PyStatus_Error("_PyPerfTrampoline_Fini() failed");
}
if (was_active) {
_PyPerfTrampoline_Init(1);
}
Expand Down
5 changes: 4 additions & 1 deletion Python/pylifecycle.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1940,7 +1940,10 @@ finalize_interp_clear(PyThreadState *tstate)
_Py_HashRandomization_Fini();
_PyArg_Fini();
_Py_ClearFileSystemEncoding();
_PyPerfTrampoline_Fini();
if (_PyPerfTrampoline_Fini() < 0) {
PyErr_FormatUnraisable("Exception ignored on "
"_PyPerfTrampoline_Fini()");
}
_PyPerfTrampoline_FreeArenas();
}

Expand Down
13 changes: 7 additions & 6 deletions Python/pystate.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2896,8 +2896,8 @@ PyGILState_Release(PyGILState_STATE oldstate)
/* Other API */
/*************/

_PyFrameEvalFunction
_PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
PyUnstable_FrameEvalFunction
PyUnstable_InterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
{
if (interp->eval_frame == NULL) {
return _PyEval_EvalFrameDefault;
Expand All@@ -2906,15 +2906,15 @@ _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
}


void
_PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
_PyFrameEvalFunction eval_frame)
int
PyUnstable_InterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
PyUnstable_FrameEvalFunction eval_frame)
{
if (eval_frame == _PyEval_EvalFrameDefault) {
eval_frame = NULL;
}
if (eval_frame == interp->eval_frame) {
return;
return 0;
}
#ifdef _Py_TIER2
if (eval_frame != NULL) {
Expand All@@ -2925,6 +2925,7 @@ _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
_PyEval_StopTheWorld(interp);
interp->eval_frame = eval_frame;
_PyEval_StartTheWorld(interp);
return 0;
}


Expand Down
Loading