.NET: Hosted agent streaming drops url_citation annotations (no response.output_text.annotation.added SSE emitted)

[graemefoster]

Description

This would re-open #6641 - unfortunately the fix did not fix the problem.

The issue is the OutputConverter class has no case statement for SSEs from the LLM which look like this:

event: response.output_text.annotation.added
data: {"type":"response.output_text.annotation.added","annotation":{"type":"url_citation","end_index":221,"start_index":209,"title":"MO_Policy_KM_Agent_v3.2.docx","url":"https://search-knowledgebaseagent.search.windows.net/"},"annotation_index":1,"content_index":0,"item_id":"msg_02959b8381e6341a006a3b6a06c39c81958191aa20a92096b2","output_index":2,"sequence_number":48}

which follow something like this:
event: response.output_text.delta
data: {"type":"response.output_text.delta","content_index":0,"delta":"【5:4†source】\n\n","item_id":"msg_02959b8381e6341a006a3b6a06c39c81958191aa20a92096b2","logprobs":[],"obfuscation":"1i","output_index":2,"sequence_number":47}

The dotnet MAF agent only outputs the response.output_text.delta event. There's no reference to the url_citation.

Unfortunately when calling the AI Search tool we still don't see any response.output_text.annotation.added SSE events coming out of the hosted agent.

Code Sample

var search = FoundryAITool.CreateAzureAISearchTool(new AzureAISearchToolOptions(
[
    new AzureAISearchToolIndex()
    {
        ProjectConnectionId = "/...azure-ai-search-connection",
        IndexName = "mortgage-operations-kb"
    }
]));

ChatClientAgentOptions options = new()
{
    Name = "KnowledgeBaseAgentOTel",
    Description = "Mortgage Operations Knowledge Agent with toolbox-powered search",
    ChatOptions = new ChatOptions
    {
        ModelId = deployment,
        Instructions = instructions,
        Tools = [search]
    },
};

var agent = projectClient
    .AsAIAgent(options);

Error Messages / Stack Traces

No error - but the Hosted Agent does not emit response.output_text.annotation.added SSE events. The events do get returned from the LLM call to the Hosted Agent.

Package Versions

Microsoft.Agents.AI - 1.13.0

.NET Version

dotnet 10.

Additional Context

No response

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

The bug is in the streaming annotation pipeline: OutputConverter in dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/OutputConverter.cs correctly handles CitationAnnotation on TextContent.Annotations (lines 122-128), but the upstream MEAI.OpenAI adapter does not populate TextContent.Annotations for response.output_text.annotation.added SSE events during streaming. The framework needs a DelegatingChatClient decorator (likely in Microsoft.Agents.AI.Foundry/FoundryChatClient.cs or a new file) that inspects the raw OpenAI SDK streaming objects for annotation events and produces TextContent with CitationAnnotation populated. Minimal repro: create a ChatClientAgent with FoundryAITool.CreateAzureAISearchTool, invoke RunStreamingAsync, and observe that AgentResponseUpdate.Contents never contains TextContent with populated Annotations for citation events.

• Failing test: none
• Files examined: dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/OutputConverter.cs, dotnet/src/Microsoft.Agents.AI.Foundry/FoundryChatClient.cs, dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs, dotnet/src/Microsoft.Agents.AI.Abstractions/AgentResponseUpdate.cs, dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/OutputConverterTests.cs, dotnet/src/Microsoft.Agents.AI/ChatClient/PerServiceCallChatHistoryPersistingChatClient.cs, dotnet/src/Microsoft.Agents.AI.Foundry/AIProjectClientExtensions.cs
• Tests run: OutputConverterTests (966 existing tests all pass), 3 custom annotation-gap tests (all pass confirming OutputConverter handles annotations correctly)
• Reported version: 1.13.0
• Current version: 1.13.0