Uh oh!
There was an error while loading. Please reload this page.
Python: Capture workflow telemetry input and output - #7565
Conversation
There was a problem hiding this comment.
Pull request overview
Adds opt-in workflow payload telemetry for improved OpenTelemetry/OpenInference observability.
Changes:
- Captures workflow, executor, and message payloads when sensitive telemetry is enabled.
- Adds message routing and OpenInference attributes.
- Tests enabled, disabled, routing, structured-output, and serialization-failure behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
python/packages/core/agent_framework/observability.py | Adds payload attributes and serialization helpers. |
python/packages/core/agent_framework/_workflows/_workflow.py | Captures workflow input and explicit outputs. |
python/packages/core/agent_framework/_workflows/_workflow_context.py | Captures sent-message content and routing. |
python/packages/core/agent_framework/_workflows/_executor.py | Captures executor input and output. |
python/packages/core/tests/workflow/test_workflow_observability.py | Adds focused telemetry tests. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
slcnx
commented
Aug 7, 2026
@microsoft-github-policy-service agree [company="yikaobang"] |
slcnx
commented
Aug 7, 2026
@microsoft-github-policy-service agree company="yikaobang" |
| if event.type == "request_info": | ||
| saw_request = True | ||
| elif workflow_outputs is not None and event.type == "output": | ||
| workflow_outputs.append(event.data) |
There was a problem hiding this comment.
Could we please avoid retaining and serializing every streamed output as one span attribute? With sensitive telemetry enabled, workflow_outputs keeps the complete output set until convergence and _set_sensitive_span_attributes then builds another JSON representation of it, so a high-volume workflow can exhaust memory solely from payload capture. How about a bounded or truncated capture to preserve observability without making successful workflow execution depend on total output volume?
Python Test Coverage Report •
Python Unit Test Overview
| ||||||||||||||||||||||||||||||||||||||||||||||||||
Evan Mattson (moonbox3)
commented
Aug 19, 2026
/review |
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (2 commit(s)): 51dc35679ae2, 7f0c8faa9820
Model:gpt-5.6-sol
Overview
The change gates payload capture on both sensitive-data enablement and recording spans, normalizes wrapped workflow inputs, and uses strict JSON with a type-only fallback on serialization errors. The focused tests cover enabled and disabled capture, routing, structured output, serialization failure, and non-finite values. Two residual risks remain: conversion hooks can mutate live payloads before execution, and outputs already delivered to consumers are omitted from workflow telemetry when execution does not converge successfully.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 2 files. Details are attached to the affected lines below.
Affected areas:python/packages/core/agent_framework/_workflows/_workflow.py, python/packages/core/agent_framework/observability.py
| def _serialize_for_telemetry(value: Any) -> str: | ||
| """Serialize heterogeneous telemetry payloads without affecting application execution.""" | ||
| try: | ||
| return json.dumps(make_json_safe(value), ensure_ascii=False, allow_nan=False) |
There was a problem hiding this comment.
make_json_safe invokes payload-defined model_dump(), to_dict(), or dict() methods on the live object. When one of those hooks is stateful, enabling sensitive telemetry mutates the payload before the workflow or executor handler receives it, so observability changes application behavior even though exceptions are caught. Please avoid invoking arbitrary conversion hooks on live application objects during telemetry capture, or otherwise isolate the conversion so handler-visible state cannot be changed.
| # Track request events for final status determination | ||
| if event.type == "request_info": | ||
| saw_request = True | ||
| elif workflow_outputs is not None and event.type == "output": |
There was a problem hiding this comment.
Output capture only runs in the successful convergence loop and the attribute is written after that loop completes. If an executor queues an output and then fails, the exception path drains and yields that output without recording it; similarly, closing the stream after an output skips finalization. This leaves output.value absent even though the caller received workflow output. Please capture drained output events as well and finalize the accumulated attribute on failure or generator close without changing the workflow's output-designation rules.
Evan Mattson (moonbox3)
commented
Aug 21, 2026
Please re-open the PR when ready to address the feedback. |
Motivation & Context
Python workflow spans currently expose structural metadata but omit the payloads processed by executors and sent between them. This prevents OpenTelemetry backends from showing useful workflow inputs and outputs.
This change closes the Python payload-capture gap identified in #3075, follows the sensitive-data behavior introduced for .NET in #3467, and adds vendor-neutral OpenInference attributes so compatible backends can render workflow I/O directly.
Description & Review Guide
Related Issue
Fixes#3075
Contribution Checklist