Skip to content

[AI Evaluation] Add versioning to reporting and caching - #6070

Merged
peterwald merged 10 commits into
mainfrom
pewaldsc/versioning
Mar 13, 2025
Merged

[AI Evaluation] Add versioning to reporting and caching#6070
peterwald merged 10 commits into
mainfrom
pewaldsc/versioning

Conversation

@peterwald

@peterwaldpeterwald commented Mar 10, 2025•
edited by dotnet-policy-service Bot
Loading

Copy link
Copy Markdown
Member

These are basically independent escape mechanisms to handle future changes in formats.

ReportFormatVersion - Will record which version of the format was used to persist the scenario run, as well as exposing which version the current library will write. This should allow us to convert/adapt to reporting format changes in the future if needed.

CacheVersion - Will change all the caching keys causing the existing cache entries to be no longer valid. @stephentoub@SteveSandersonMS

Hopefully neither of these will ever be needed.

Microsoft Reviewers: Open in CodeFlow

@peterwaldpeterwald added area-ai Microsoft.Extensions.AI libraries area-ai-eval Microsoft.Extensions.AI.Evaluation and related labels Mar 10, 2025
@peterwaldpeterwald self-assigned this Mar 10, 2025
@peterwald
peterwald requested review from a team as code ownersMarch 10, 2025 20:34
@RussKieRussKie removed the area-ai Microsoft.Extensions.AI libraries label Mar 10, 2025
stephentoub
stephentoub reviewed Mar 11, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI/ChatCompletion/DistributedCachingChatClient.cs Outdated
@peterwald

peterwald commented Mar 11, 2025

Copy link
Copy Markdown
MemberAuthor

@dotnet/dotnet-extensions-ai This change is ready to merge if you approve.

RussKie
RussKie reviewed Mar 11, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs Outdated
RussKie
RussKie reviewed Mar 11, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs Outdated
peterwaldand others added 2 commits March 11, 2025 18:29
@peterwald@RussKie
Update src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSh…
f332da2
…arp/ScenarioRunResult.cs
Co-authored-by: Igor Velikorossov <RussKie@users.noreply.github.com>
@peterwald@RussKie
Update src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSh…
0883cc0
…arp/Defaults.cs
Co-authored-by: Igor Velikorossov <RussKie@users.noreply.github.com>
stephentoub
stephentoub reviewed Mar 12, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs Outdated
stephentoub
stephentoub reviewed Mar 12, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs Outdated
stephentoub
stephentoub reviewed Mar 12, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs Outdated
shyamnamboodiripad
shyamnamboodiripad reviewed Mar 12, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs Outdated
@peterwald

peterwald commented Mar 13, 2025

Copy link
Copy Markdown
MemberAuthor

Updated based on your feedback @shyamnamboodiripad@stephentoub

stephentoub
stephentoub reviewed Mar 13, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs Outdated
stephentoub
stephentoub reviewed Mar 13, 2025
View reviewed changes
Comment threadsrc/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs Outdated
@peterwald
peterwald merged commit e393f33 into mainMar 13, 2025
@peterwald
peterwald deleted the pewaldsc/versioning branch March 13, 2025 18:29
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 13, 2025
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@shyamnamboodiripadshyamnamboodiripadshyamnamboodiripad approved these changes
@stephentoubstephentoubstephentoub approved these changes
+1 more reviewer
@RussKieRussKieRussKie left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@peterwaldpeterwald

Labels

area-ai-evalMicrosoft.Extensions.AI.Evaluation and related

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@peterwald@stephentoub@eiriktsarpalis@RussKie@shyamnamboodiripad