Add ToChatCompletion{Async} methods for combining StreamingChatCompletionUpdates

[stephentoub]

WithMessageAddedAsync enables writing code like:

List<ChatMessage>messages= ...;awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).WithMessageAddedAsync(messages)){Console.Write(update);}

and upon completion of the loop, messages will contain a ChatMessage merged from all of the updates.

ToChatCompletion is what's used to achieve that, but is also exposed on its own so that code which has an enumerable of StreamingChatCompletionUpdates can merge them into a ChatCompletion. This enables a non-streaming implementation to easily be authored in terms of a streaming one.

Microsoft Reviewers: Open in CodeFlow

[eiriktsarpalis]

WithMessageAddedAsync enables writing code like:

I hadn't realized that CompleteStreamingAsync doesn't update the chat history after the IAE has been enumerated. I'm guessing that this is influenced by the deferred execution of the resultant IAE, but it does seem possible we could go the other way and require coalescence to be a part of CompleteStreamingAsync for consistency with CompleteAsync.

[stephentoub]

WithMessageAddedAsync enables writing code like:

I hadn't realized that CompleteStreamingAsync doesn't update the chat history after the IAE has been enumerated. I'm guessing that this is influenced by the deferred execution of the resultant IAE, but it does seem possible we could go the other way and require coalescence to be a part of CompleteStreamingAsync for consistency with CompleteAsync.

Neither of them update the chat history with the resulting message.

[SteveSandersonMS]

The naming of WithMessageAddedAsync feels odd. Phrasing it in the past tense sounds as if the message has already been added in some way, but it adds the message in the future after the streaming completes.

Do any of the following feel right to you?

awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).AppendMessageTo(messages))awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).AppendResultTo(messages))awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).AppendResponseTo(messages))awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).WithAppendTo(messages))awaitforeach(varupdateinclient.CompleteStreamingAsync(messages).ThenAppendTo(messages))

I mildly lean towards AppendResultTo or AppendResponseTo but don't feel strongly.

[eiriktsarpalis]

I tend to associate the Append prefix with methods that return void; With more accurately reflects we're decorating the underlying IAE with behaviour; Then is typically used when appending callbacks in the style of Task.ContinueWith which isn't precisely what this method is doing.

[stephentoub]

The hard part here is the ToChatCompletion. With that, someone can do the equivalent of WithMessageAddedAsync with:

List<StreamingChatCompletionUpdate>updates=[];awaitforeach(varupdateinclient.CompleteStreamingAsync(messages)){Console.Write(update);updates.Add(update);}messages.Add(updates.ToChatCompletion().Message);

Given that all of the raised concerns are about WithMessageAddedAsync, I'll just remove that for now.

[stephentoub]

This is now just:

publicstaticChatCompletionToChatCompletion(thisIEnumerable<StreamingChatCompletionUpdate>updates,boolcoalesceContent=true)

[stephentoub]

(though now that I write that out, I'm wondering if I should add one for IAsyncEnumerable as well)

[stephentoub]

Refactored to have ToChatCompletion{Async} and to have them in the Abstractions library. Having them there means a leaf client can choose to use e.g. ToChatCompletionAsync to implement its CompleteAsync method around its CompleteStreamingAsync, if desired.

[eiriktsarpalis]

As Microsoft.Extensions.AI.Abstractions is accumulating more and more helper methods, I'm growing somewhat uncomfortable with the "Abstractions" suffix in this project. Would it make sense to rename this to be just Microsoft.Extensions.AI and then have the existing Microsoft.Extensions.AI called something along the lines of Microsoft.Extensions.AI.Middlewares?

cc @SteveSandersonMS

[stephentoub]

As Microsoft.Extensions.AI.Abstractions is accumulating more and more helper methods, I'm growing somewhat uncomfortable with the "Abstractions" suffix in this project. Would it make sense to rename this to be just Microsoft.Extensions.AI and then have the existing Microsoft.Extensions.AI called something along the lines of Microsoft.Extensions.AI.Middlewares?

cc @SteveSandersonMS

I think it's fine. These helpers are to aid in the development of implementations of the abstractions. It's very much in line with other M.E libs.

[SteveSandersonMS]

Looks good.

In a previous iteration of this PR, I think you were able to remove the equivalent logic from CachingChatClient and use your new helper instead. But in the current version of the PR that's no longer the case. Is it no longer possible to avoid having two versions of this logic?

[stephentoub]

Is it no longer possible to avoid having two versions of this logic?

I put these into m.e.ai.abstractions, and the caching stuff is up in m.e.ai. We could reconsolidate if we wanted to expose the coalescing publicly from abstractions.

[stephentoub]

Is it no longer possible to avoid having two versions of this logic?

I put these into m.e.ai.abstractions, and the caching stuff is up in m.e.ai. We could reconsolidate if we wanted to expose the coalescing publicly from abstractions.

Went another way: #5616