Python: fix streaming when GenAI tracing replaces the raw response (#7461)

[madanmishra1223]

Fixes#7461

Problem

With AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true, every streaming request fails:

AttributeError: 'AsyncStreamWrapper' object has no attribute 'parse'

The Azure GenAI instrumentor replaces the SDK's raw-response wrapper with an object that is the event stream and exposes neither .parse() nor .headers.

Both streaming paths in _chat_client.py already anticipate this for headers, and say so in a comment:

# Read headers defensively: telemetry instrumentors (e.g. azure-ai-projects# experimental tracing) wrap the streaming response in objects that do not# proxy ``.headers``. Degrade gracefully so the served-model surfacing is# best-effort instead of crashing the whole call.served_model=self._extract_served_model(getattr(raw_stream_response, "headers", None))
asyncwithraw_stream_response.parse() asstream_response: # <-- not defensive

.headers is read through getattr, then .parse() is called unconditionally on that same object. So the defense is half-applied: the attribute that only degrades a nice-to-have is guarded, and the attribute that breaks the entire call is not.

Fix

Read .parse defensively too. _open_event_stream() uses .parse() when present — preserving the async with so the socket still closes deterministically — and otherwise iterates the object directly, letting the instrumentor own the stream's lifetime.

Behavior is unchanged for the normal SDK wrapper; only the instrumented case changes, from raising to streaming.

Both affected streaming call sites are updated (the retrieve continuation path and the create path).

The two non-streaming .parse() sites are deliberately left alone: the instrumentor's wrapper is stream-specific (AsyncStreamWrapper), and non-streaming was not reported as failing. Happy to extend if maintainers prefer symmetry.

Verification

Repro before the fix, using a stand-in wrapper that is async-iterable with no .parse()/.headers:

instrumented object: AsyncStreamWrapper
has .parse() : False
has .headers : False
async-iterable: True
BUG: ChatClientException: (... service failed to complete the prompt:
'AsyncStreamWrapper' object has no attribute 'parse'", AttributeError(...))

After:

RESULT: streamed 'Hello world'
OK: streaming survived the instrumented wrapper

Added test_streaming_survives_instrumented_response_without_parse, which fails on main with the exact reported AttributeError and passes with this change.

Checks run locally:

Check	Result
pytest packages/openai/tests	453 passed, 89 skipped
pytest packages/core/tests	4080 passed, 23 skipped, 2 xfailed
pyright (openai package)	0 errors — same as baseline
mypy (test file)	clean
ruff format --check / ruff check	clean

Note

The issue is assigned to Tao Chen (@TaoChenOSU). There was no linked PR after ~2.5 weeks, so I picked it up — happy to close this if it is already in progress.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a defensive streaming path so chat streaming continues to work when telemetry/tracing replaces the SDK raw-response wrapper (removing .parse() / .headers), and introduces a regression test for that scenario.

Changes:

• Added _open_event_stream() async context manager to safely obtain an event stream with/without .parse().
• Updated streaming code paths to use _open_event_stream() instead of unconditionally calling .parse().
• Added a regression test that simulates an instrumented bare async stream without .parse() / .headers.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
python/packages/openai/agent_framework_openai/_chat_client.py	Adds _open_event_stream() and switches streaming to use it to avoid .parse() AttributeError under instrumentation.
python/packages/openai/tests/openai/test_openai_chat_client.py	Adds regression coverage for streaming when the raw response is replaced by a bare async stream.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[madanmishra1223]

Thanks both — Evan Mattson (@moonbox3) you were right, and the PR is updated.

Verified the failure you described

I reproduced it against the real instrumentor (azure-ai-projects==2.3.0, openai==2.53.0, AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true), with a mocked transport so no Azure resources are needed. The instrumentor needs both an OTel TracerProviderandazure-core-tracing-opentelemetry for _create_responses_span_from_parameters to return a span — without them it returns the result unwrapped and the bug doesn't appear at all, which is what made this easy to miss.

With tracing genuinely active:

[wrap] _wrap_async_streaming_response receives stream=openai._legacy_response.LegacyAPIResponse
with_raw_response.create() -> azure.ai.projects.telemetry._responses_instrumentor.AsyncStreamWrapper
has .parse() : False
has .headers : False
.stream_async_iter = openai._legacy_response.LegacyAPIResponse
has __anext__: False has parse: True
--- iterating the wrapper directly (what the first commit did) ---
FAILS: AttributeError: 'LegacyAPIResponse' object has no attribute '__anext__'

Exactly as you said: with_raw_response.create() routes through the instrumented AsyncResponses.create, so stream_async_iter is the still-unparsed LegacyAPIResponse. My first commit just traded one AttributeError for another and traced streaming stayed broken.

You were also right about why the test missed it: it patched with_raw_response.create, which is above the layer that installs the wrapper, so the wrapper's inner raw response never existed in that test.

What changed

_open_event_stream() now parses the raw response the wrapper holds and hands it back, so the wrapper stays in the iteration path and keeps recording telemetry:

inner=getattr(raw_response, "stream_async_iter", None)
inner_parse=getattr(inner, "parse", None)
ifcallable(inner_parse):
asyncwithcast("Any", inner_parse()) asstream:
raw_response.stream_async_iter=streamyieldraw_responsereturn

End-to-end against the real instrumentor now:

raw from with_raw_response : AsyncStreamWrapper
.stream_async_iter : LegacyAPIResponse
_open_event_stream yielded: AsyncStreamWrapper
RESULT: streamed 'Hello world'
telemetry wrapper still in iteration path: True

The test now models that observed object graph — a wrapper with no parse/headers whose stream_async_iter is an unparsed raw response — and fails against my previous commit with AttributeError: '_UnparsedRawResponse' object has no attribute '__aiter__', mirroring the real failure. I added a second test for a bare event stream with nothing to parse, covering the remaining branch.

I kept the fake rather than adding azure-ai-projects as a test dependency, since that would be a heavy dep for this package — happy to add a marked integration test instead if you'd prefer the real thing in CI.

On the two Copilot comments

• callable(parse) — good catch, adopted for both the outer and inner lookups.
• AsyncIterator vs AsyncGenerator — respectfully leaving this as AsyncGenerator. This repo's pinned pyright (1.1.411) reports the opposite as an error: Annotating the return type as -> AsyncIterator[Foo] with @asynccontextmanager is deprecated. Use -> AsyncGenerator[Foo] instead. (reportDeprecated). I had it as AsyncIterator first and pyright failed on it; AsyncGenerator is what keeps the package at 0 errors.

Checks

Check	Result
pytest packages/openai/tests	454 passed, 89 skipped
pytest packages/core/tests	4080 passed, 23 skipped, 2 xfailed
pyright (openai package)	0 errors — matches baseline
mypy (test file)	clean
ruff format --check / ruff check	clean
e2e vs real azure-ai-projects==2.3.0	streams correctly, wrapper retained

[Evan Mattson (moonbox3)]

madanmishra1223 please resolve all open comments if they've been addressed

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/openai/agent_framework_openai
_chat_client.py	1405	119	91%	361, 374, 728–734, 743–746, 752–756, 764, 808–813, 817–820, 822–824, 831–833, 836, 893, 901, 924, 1141, 1200, 1202, 1204, 1206, 1272, 1286, 1366, 1376, 1381, 1424, 1536–1537, 1552, 1831, 1938, 1943–1944, 2027, 2037, 2064, 2070, 2080, 2086, 2091, 2097, 2102–2103, 2183, 2227, 2230–2233, 2247, 2257–2258, 2270, 2312, 2377, 2394, 2397, 2424–2426, 2465, 2482, 2485, 2547, 2554, 2591–2592, 2627, 2665–2666, 2684–2685, 2857–2858, 2876, 2962–2970, 3148, 3163, 3252–3254, 3264–3265, 3271, 3286, 3419–3420
TOTAL	47141	4373	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9563	36 💤	0 ❌	0 🔥	2m 36s ⏱️

[Tao Chen (TaoChenOSU)]

Hi madanmishra1223,

Thanks for the contributions! Could you also create an issue in the Azure SDKs repo and link it here?

[madanmishra1223]

Thanks Tao Chen (@TaoChenOSU) — filed upstream: Azure/azure-sdk-for-python#48646

[azure-ai-projects] ResponsesInstrumentor breaks openai with_raw_response streaming (AsyncStreamWrapper wraps an unparsed LegacyAPIResponse)

It has a standalone repro (openai + azure-ai-projects only, mocked transport, no Azure resources) and frames it as the raw-response contract being broken by the instrumentor:

• with_raw_response.create(..., stream=True) returns AsyncStreamWrapper instead of a raw response
• no .parse(), no .headers
• stream_async_iter is the still-unparsed LegacyAPIResponse, so iterating raises AttributeError: 'LegacyAPIResponse' object has no attribute '__anext__'
• uninstrumenting restores the correct LegacyAPIResponse with .parse() / .headers

I called out there that reproducing needs both an OTel TracerProviderandazure-core-tracing-opentelemetry — without them _create_responses_span_from_parameters returns None, the instrumentor returns the result unwrapped, and the bug silently doesn't appear. That tripped me up initially, so it seemed worth stating explicitly for whoever picks it up.

I also linked this PR from that issue and noted the workaround here can be dropped once it's fixed upstream.

One process note: this branch is still showing action_required on the workflows, so the Test Typing Checks fix in c85069cd hasn't been re-verified by CI yet. It passes locally via the task CI uses (poe test-typing -P openai: mypy, pyrefly, ty, zuban, pyright all green) — but it would need a workflow approval to confirm in CI.

[Tao Chen (TaoChenOSU)]

Hi madanmishra1223,

Thanks! Please also resolve all open comments when you can.

[madanmishra1223]

Tao Chen (@TaoChenOSU) all three review threads are resolved. Since resolving collapses them, here is what each one says, so it is readable without expanding:

1. Copilot — guard parse with callable() (r3799495111)
Adopted. Both lookups now use callable() instead of an is None check, which matters more than it first looked since the helper probes parse on two different objects:

parse=getattr(raw_response, "parse", None)
ifcallable(parse): ...
inner_parse=getattr(inner, "parse", None)
ifcallable(inner_parse): ...

2. Copilot — use AsyncIterator instead of AsyncGenerator (r3799495126)
Not applied, deliberately. This repo's pinned pyright reports the opposite as an error. I had it as AsyncIterator first and pyright==1.1.411 failed the package:

Annotating the return type as `-> AsyncIterator[Foo]` with `@asynccontextmanager` is deprecated.
Use `-> AsyncGenerator[Foo]` instead. (reportDeprecated)

Baseline for the package is 0 errors; AsyncIterator took it to 1. Switching back would break Test Typing Checks. Happy to revisit if you would rather match the convention and suppress the rule.

3. Evan Mattson (@moonbox3) — exercise it through the real azure-ai-projects instrumentor (r3800092642)
You were right, and this changed the fix. Reproduced against azure-ai-projects==2.3.0 + openai==2.53.0 with a mocked transport: with_raw_response.create() returns AsyncStreamWrapper whose stream_async_iter is the still-unparsed LegacyAPIResponse, so my first commit only traded one AttributeError for another:

AttributeError: 'LegacyAPIResponse' object has no attribute '__anext__'

Your point about the test was right too — it patched with_raw_response.create, above the layer that installs the wrapper, so the inner raw response never existed in it.

_open_event_stream() now parses the wrapped raw response and hands it back, keeping the wrapper in the iteration path so telemetry still records. End-to-end against the real package:

raw from with_raw_response : AsyncStreamWrapper
.stream_async_iter : LegacyAPIResponse
_open_event_stream yielded: AsyncStreamWrapper
RESULT: streamed 'Hello world'
telemetry wrapper still in iteration path: True

The test now models that object graph and fails against my previous commit.

---

Status: upstream issue filed and linked (Azure/azure-sdk-for-python#48646), branch up to date with main, and CI is green on the current head — 35 checks, 23 success / 12 skipped / 0 failed, including the Test Typing Checks job that was previously failing.