Python: add MiddlewareFailure, a first-class fatal signal for function middleware

[MohammadHaroonAbuomar]

Motivation & Context

_auto_invoke_function converts every exception raised by function middleware (or the tool it wraps) into a tool-error result and keeps looping, so the only loud escape from the auto-invoke loop is MiddlewareTermination. Middleware that needs fail-closed semantics — enforcement layers, guardrails — had to mutate shared state, raise the loop's one loud exception, and re-raise the real failure two hops away at the run boundary, which is exactly what the agent-hooks feature (#7515) did with its _RunState.halted back-channel. Evan Mattson (@moonbox3) flagged this in review (#7515 (comment)) and suggested tracking a first-class signal as follow-up; #7522 is that follow-up.

This PR gives every function-middleware author fail-closed semantics without the state-mutation dance, and migrates the agent-hooks feature onto the new signal, deleting the back-channel.

Description & Review Guide

• What are the major changes?

  1. New public exception MiddlewareFailure(MiddlewareException) (_middleware.py, exported like MiddlewareTermination): the function-invocation loop's explicit fail-closed escape. Raise it from function middleware (or a tool body) to abort the run; chain the underlying error with raise ... from. Middleware must not catch it — swallowing it converts a fail-closed abort back into a running, possibly unguarded loop.

  2. The _tools.py loop contract widens, precisely as follows. Before: every exception raised by function middleware or a tool body — except MiddlewareTermination and UserInputRequiredException — was converted into a tool-error function_result and the loop continued. After: exactly one additional exception type, the new explicit MiddlewareFailure, is re-raised instead of absorbed, on both the direct and the middleware-pipeline execution paths; when it escapes one call of a parallel batch, the in-flight sibling invocations are cancelled and awaited before it propagates out of get_response to the run's caller (for streaming runs, it is raised when the stream is consumed). Cancellation is cooperative: an async sibling stops at its next suspension point, while a synchronous tool body already executing in a worker thread (asyncio.to_thread) cannot be interrupted and may complete its side effects — its result is discarded either way and never reaches the transcript, the model, or history, and failure propagation is not delayed behind it (pinned by a blocking-sync-sibling regression test). On a service-managed conversation (persisted conversation id), the loop settles the aborted batch before propagating — one error function_result per dangling call (approval-response wrappers unwrap to their underlying calls; hosted-tool approvals are left to their own provider protocol), submitted with tool_choice="none" in a single extra request — and then advances the persisted continuation to the settlement response (required for response-ID continuations such as OpenAI Responses store=True, where the settlement response is the first endpoint whose chain includes the synthetic outputs; a no-op for stable conversation-object ids); the settlement response is otherwise discarded. Settlement also covers the approval-resolution phase, so a failure raised while an approved tool is replayed settles the original, already-persisted calls. Without a service-managed conversation no extra request is made. (Raised by Evan Mattson (@moonbox3) in review, pinned by six regression tests across both response modes.) The absorb-into-tool-error contract for ordinary exceptions is unchanged and now pinned by a dedicated regression test; MiddlewareTermination semantics are untouched. As a drive-by, the new cancel-and-await around the batch gather also stops sibling tool tasks from being orphaned on the pre-existing missing-call_idKeyError escape path.

  3. Why a dedicated exception type instead of a fatal flag on MiddlewareTermination (the two shapes proposed in Python: a first-class fatal signal for function middleware (fail-closed escape from the auto-invoke loop) #7522; Atharva Vichare (@atty57) asked the same question on the issue, including whether the agent/chat seams should honor it):

  • The repo has six places that absorb or specially handle MiddlewareTermination (agent pipeline suppress, chat pipeline suppress, two _tools.py sites, the harness loop, purview). A flag fails open at any site — present or future — that forgets to check it; a distinct type bypasses all of them by construction. Empirically: harness and purview needed zero changes.
  • Mutation probe: re-basing MiddlewareFailure onto MiddlewareTermination (simulating flag-style handling) makes 7 of the 8 new regression tests fail silently open — the runs complete normally with the fatal signal absorbed at the various termination-handling seams. Only the absorb-contract pin still passes. That settles type-vs-flag empirically.
  • Agent and chat middleware already have fail-loud semantics for every exception, and MiddlewareFailure behaves identically there, so one type gives uniform semantics across all three categories with no per-seam code (pinned by test).
  • It also sidesteps the result-capture machinery _auto_invoke_function performs on terminations, which a fatal termination would have to special-case.

  4. Agent-hooks migration (the Python: a first-class fatal signal for function middleware (fail-closed escape from the auto-invoke loop) #7522 deliverable "delete the halted back-channel and the approval special case"):

  • Deleted: _RunState.halted and all three run-boundary raise state.halted checks; the state.halted arm of the termination special case in the function middleware; the context.result mutation in _halt_on_enforcement_failure; the no-run-state terminate-with-error-result hack (a partial install now fails loudly with MiddlewareFailure).
  • The approval-request pass-through is retained but consolidated to a single check on the normal path: it is framework approval control flow independent of failure signaling, and deleting it outright would swallow third-party MiddlewareTermination and change should_terminate on the approval-replay path. It is pinned by a new termination-path approval test.
  • Tool-seam host_error blocks keep surfacing as agent_hooks.InterceptionBlocked at the run boundary (one deny surface at every seam, matching the model-seam behavior): the feature's own halts travel as a privately tagged MiddlewareFailure subclass whose chained InterceptionBlocked cause is re-raised by the agent middleware. Only the private tag authorizes the unwrap — a third-party MiddlewareFailure with a crafted InterceptionBlocked cause propagates exactly as raised, so untrusted middleware cannot launder an attacker-shaped interception record into the feature's audit-bearing deny surface (pinned by an adversarial regression test, verified by mutation).

  5. Spec and docs: docs/specs/004-python-function-calling-loop.md gains the middleware-failure invariants, two scenario-matrix rows naming the regression tests, and the related-issue entry; python/CODING_STANDARD.md exception tree updated; MiddlewareFailure and FunctionMiddleware docstrings document the exception semantics.

• What is the impact of these changes?

  Non-breaking. No existing behavior changes for code that does not raise the new exception: ordinary exceptions still absorb into tool errors, MiddlewareTermination is untouched, and the non-hooked loop behavior is pinned by regression tests. Within the experimental agent-hooks feature, two deliberate surface improvements: a partial bundle install now fails loudly instead of quietly stopping the loop, and enforcement-layer failures surface as MiddlewareFailure (still a MiddlewareException, so existing handlers keep working).

• What do you want reviewers to focus on?

  The exact wording of the widened loop contract in item 2 (it is the load-bearing behavior change), the batch cancellation semantics in _try_execute_function_call_groups, and the tagged-unwrap design at the agent-hooks run boundary.

Related Issue

Fixes#7522

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework
_agent_hooks.py	759	149	80%	273, 310, 313–316, 318–329, 332–333, 339–340, 367, 371, 404, 406, 414–415, 421, 426, 469, 487–488, 562, 571–572, 578–579, 586, 590, 601–602, 605–606, 610, 622–623, 626, 629, 639–641, 643–644, 650, 657–665, 668–669, 676–679, 681–682, 707, 711, 739–745, 748–750, 774–781, 783–784, 803, 805, 814, 816–821, 831, 842–845, 861, 872–881, 1037–1038, 1098–1099, 1153, 1171, 1173–1174, 1202–1203, 1226, 1238–1243, 1311–1312, 1357, 1411, 1434, 1447, 1457, 1460–1464, 1487
_middleware.py	443	16	96%	435, 808, 815–816, 998, 1069, 1087, 1089, 1222, 1225, 1252, 1254, 1390, 1394, 1581, 1657
_tools.py	1528	100	93%	232–233, 410, 412, 425, 450–452, 460, 478, 492, 499, 506, 529, 531, 538, 546, 681, 715–717, 720–722, 724, 730, 781–783, 808, 834, 838, 876–878, 882, 1055, 1067, 1074–1077, 1098, 1106, 1120–1122, 1496, 1581, 1694–1695, 1752, 1799, 1806–1807, 1911, 1952–1953, 1988, 2084, 2098, 2101, 2108, 2111, 2117, 2129, 2146, 2155, 2163, 2167, 2187, 2189, 2196, 2254, 2257, 2280, 2287, 2292–2293, 2296, 2300, 2303, 2325, 2359, 2427, 2456–2457, 2554, 2582, 2622, 2625, 2682, 2835, 2944, 3117, 3120, 3130, 3147–3148, 3616
TOTAL	46873	4362	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9486	36 💤	0 ❌	0 🔥	2m 33s ⏱️

[Copilot]

Pull request overview

Adds MiddlewareFailure as a public fail-closed signal for function middleware and tools.

Changes:

• Propagates fatal middleware failures and cancels parallel tool tasks.
• Migrates agent-hooks away from its halted-state back-channel.
• Adds exports, documentation, specifications, and regression tests.

Show a summary per file

File	Description
python/packages/core/agent_framework/_middleware.py	Defines and documents MiddlewareFailure.
python/packages/core/agent_framework/_tools.py	Propagates failures and cancels tool batches.
python/packages/core/agent_framework/_agent_hooks.py	Migrates agent-hooks failure handling.
python/packages/core/agent_framework/__init__.py	Adds the runtime export.
python/packages/core/agent_framework/__init__.pyi	Adds the typing export.
python/packages/core/tests/core/test_middleware_with_agent.py	Tests fatal-signal behavior.
python/packages/core/tests/core/test_agent_hooks.py	Tests migrated agent-hooks behavior.
python/CODING_STANDARD.md	Updates the exception hierarchy.
docs/specs/004-python-function-calling-loop.md	Documents loop invariants and scenarios.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 2
• Review effort level: Balanced

[Evan Mattson (moonbox3)]

MohammadHaroonAbuomar please also have a look at the merge conflict