Make the `PY_UNWIND` monitoring event local

[P403n1x87]

Feature or enhancement

Proposal:

I would like to propose making the PY_UNWIND monitoring event available as a local event. The rationale for doing this is that some tools can be interested in trapping per-function exit events. Currently, this is only possible with the PY_RETURN event when a function returns with a value. However, if a function is exited with an exception, there is no way of capturing that event on just that function. Making the PY_UNWIND event would allow monitoring tools to catch all function exit events.

import random, sys

m, e = sys.monitoring, sys.monitoring.events

m.use_tool_id(0, "debugger")

def monitor(event):
 def _(f):
 m.register_callback(0, event, f)
 return _

@monitor(e.PY_START)
def _(*_, **__):
 print("entering", sys._getframe(1))

@monitor(e.PY_UNWIND)
def _(*_, **__):
 print("exiting ", sys._getframe(1), "with exception")

def foo():
 raise RuntimeError()

def bar():
 return foo()

def baz():
 return bar()

m.set_local_events(0, random.choice([foo, bar, baz]).__code__, e.PY_UNWIND | e.PY_START)

try:
 baz()
except Exception:
 pass

# entering <frame at 0x101602f80, file 'test_py_unwind_local.py', line 34, code baz>
# exiting <frame at 0x101602f80, file 'test_py_unwind_local.py', line 35, code baz> with exception

A debugger is an example of a tool that can benefit from PY_UNWIND being a local event.

The proposed change will break the current assumption that local events always require bytecode instrumentation. Instead, the PY_UNWIND event will simply make a local tool check instead of a global one inside monitor_unwind. No changes to bytecode or any other parts are required.

This is an extension of the current implementation of PEP 669 whereby PY_UNWIND will be available as a local event. The behaviour of the already existing global event will not be touched. As such, the proposed change will be guaranteed to be backwards-compatible.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/pep-669-low-impact-monitoring-for-cpython/13018/82

Linked PRs

• gh-142186: make PY_UNWIND available as a local event #142179
• gh-142186: all-local PEP-669 events #146182
• gh-142186: Revert the unintended value change in the PY_MONITORING_EVENT_* values from gh-146182 #148955

[markshannon]

It should be possible to disable local events, and disabling is specific to the location passed to the callback.
https://docs.python.org/3/library/sys.monitoring.html#local-events
How would you implement that?

Or do you want to make PY_UNWIND its own category, that can be turned on or off per code objects, but cannot be disabled?

If we are breaking the distinction between "local" meaning specific to a location and can be disabled, and "local" meaning can be turned on or off per code object, then we might as well make all events "local" in the second sense.

[P403n1x87]

It should be possible to disable local events, and disabling is specific to the location passed to the callback. https://docs.python.org/3/library/sys.monitoring.html#local-events How would you implement that?

My understanding is that DISABLE is intended for those local events that can trigger at multiple locations. It can be used to fine-tune the events that the user wants to trap (e.g. just a sub-set of all the lines within a code object, or certain return statements). PY_UNWIND is obviously an event that is not local in the sense of coming from a certain bytecode offset. However, one might argue that the event itself happens when the execution has reached a certain offset, so it (and in fact any event) is local in this other sense. But because these events do not instrument the bytecode, they cannot be DISABLEd like other local events.

Or do you want to make PY_UNWIND its own category, that can be turned on or off per code objects, but cannot be disabled?

Because as of now all local events require instrumentation, I would be more inclined to reflect the "new" meaning of local in the documentation. That is, for events that can trigger at pre-defined locations (think compile-time), returning DISABLE will do the current job, otherwise it is ignored. For example, PY_START is a local event with instrumentation at a guaranteed single location. Returning DISABLE from its handler would undo the instrumentation, which is effectively equivalent to turning the event off on the code object. So we already have a local event in its own category.

If we are breaking the distinction between "local" meaning specific to a location and can be disabled, and "local" meaning can be turned on or off per code object, then we might as well make all events "local" in the second sense.

Ideally I would like the idea of having all the events local on a per-code object basis. I haven't looked at the other events, but the PY_UNWIND one was easy enough to implement at potentially no performance cost. If it is equally convenient to turn the other currently non-local events into local ones in this new sense then I think it would be great.

[P403n1x87]

@markshannon for the sake of making progress here, would it be acceptable to merge the proposed change that makes PY_UNWIND "local" (and ignores DISABLE), and follow that up with potentially a more comprehensive change that makes all other events local in this new sense, and restrict the relevance of DISABLE to only some of the events? As I mentioned in my previous comment, the PY_START event has a similar behaviour. The fact that it can be disabled but not turned off is just a mere consequence of the implementation (which I think is hacky anyway, as shown by the "funny" coding required to handle that case in #142136), since the instrumentation for that event takes place in just a single place, regardless of the bytecode content.

[P403n1x87]

Gentle ping. I would really like for this change to make it into 3.15 if possible 🙏

[P403n1x87]

@markshannon apologies for the direct ping but I would love to resume discussing this as it is blocking our plans to move to the low-impact monitoring API for our instrumentation needs 🙏

[markshannon]

Sorry for taking so long to get back to this.

Rather than making PY_UNWIND local, what I think would be a better solution and meet your needs, is this:

• Make "global" events per code-objects, so that they can all be set per code object
• Allowing DISABLEing of "global" events, which would disable them per code object, not per location.

This should only take a little bit of extra space per code object and have no impact on performance (at least not for the uninstrumented case). We might need to change the name "global" and the docs will definitely need updating.

How does that sound?

[markshannon]

OOI, why do you need per code object unwind events. Would filtering a global event be too expensive?

[P403n1x87]

Sorry for taking so long to get back to this.

No problem, thanks for taking another look at this 🙂 .

How does that sound?

I totally agree on all points. However, just a note that our priority is just with PY_UNWIND as it's the only non-local event that would allow us to migrate to PEP-669. This is the least effort that would take us to where we'd like to be. I'll try to get a sense of the work required if the implementation should fall on our side.

why do you need per code object unwind events

At Datadog we provide dynamic instrumentation capabilities on live systems whereby we inject line probes and wrap functions to capture arguments/locals/return values/exceptions. For applications that throw exceptions for control flow, the cost of running a trap for every code object only to filter out just a few (where a probe is actually set) would be too expensive, so we would really like to target just the code objects we care about.

[P403n1x87]

@markshannon I have opened #146182 to make all the remaining events local per-code object.