Skip to content

.NET: Add Foundry Evaluation samples (Safety + Quality) - #3697

Merged
Roger Barreto (rogerbarreto) merged 11 commits into
mainfrom
copilot/support-foundry-observability
Feb 18, 2026
Merged

.NET: Add Foundry Evaluation samples (Safety + Quality)#3697
Roger Barreto (rogerbarreto) merged 11 commits into
mainfrom
copilot/support-foundry-observability

Conversation

CopilotAI commented Feb 5, 2026•
edited by rogerbarreto
Loading

Copy link
Copy Markdown
Contributor

Motivation and Context

Adds .NET evaluation samples for Foundry Agents, achieving parity with existing Python evaluation samples. Part of #3675 / #3440.

Description

This PR adds two new evaluation samples demonstrating how to assess agent safety and quality using Microsoft.Extensions.AI.Evaluation packages with Azure AI Foundry.

New Samples

SampleDescription
FoundryAgents_Evaluations_Step01_RedTeamingSafety evaluation using ContentHarmEvaluator, ViolenceEvaluator, HateAndUnfairnessEvaluator, ProtectedMaterialEvaluator, IndirectAttackEvaluator
FoundryAgents_Evaluations_Step02_SelfReflectionQuality evaluation with self-reflection loop using GroundednessEvaluator, RelevanceEvaluator, CoherenceEvaluator

Key Changes

  • New evaluation samples following the FoundryAgents_Evaluations_StepXX_* naming convention (same level as other FoundryAgents samples)
  • Added evaluation NuGet packages to Directory.Packages.props:
    • Microsoft.Extensions.AI.Evaluation 10.3.0
    • Microsoft.Extensions.AI.Evaluation.Quality 10.3.0
    • Microsoft.Extensions.AI.Evaluation.Safety 10.3.0-preview
  • Added projects to slnx solution file
  • Code conventions applied: explicit types (no var), collection expressions, DefaultAzureCredential, dotnet format clean
  • Robustness: try/finally blocks ensure agent cleanup even on evaluation failures
  • Step02 env var separation: AZURE_OPENAI_DEPLOYMENT_NAME for evaluator model (may differ from Foundry agent model)

Environment Variables

VariableUsed ByDescription
AZURE_FOUNDRY_PROJECT_ENDPOINTBothFoundry project endpoint
AZURE_FOUNDRY_PROJECT_DEPLOYMENT_NAMEBothModel for agent creation (default: gpt-4o-mini)
AZURE_OPENAI_ENDPOINTStep02Azure OpenAI endpoint for quality evaluators
AZURE_OPENAI_DEPLOYMENT_NAMEStep02Model for evaluator LLM (falls back to agent model)

Regional Requirements

  • Step01 (Safety): Requires a region that supports content harm evaluation (East US 2, Sweden Central, France Central, US North Central, Switzerland West)
  • Step02 (Quality): Works in any region with Azure OpenAI deployment

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • dotnet format passes with no changes
  • Step02 verified end-to-end (Groundedness 5.0/5, Relevance 5.0, Coherence 4.0)
  • Step01 safety evaluators require a supported region for full testing
  • Is this a breaking change? No

CopilotAI changed the title [WIP] Add .NET samples for Foundry observability and evaluations.NET: Add Foundry Evaluation samples for Red Teaming and Self-ReflectionFeb 5, 2026
CopilotAI changed the title .NET: Add Foundry Evaluation samples for Red Teaming and Self-ReflectionRefactor evaluation samples: Replace instructional Console.WriteLine with real implementationsFeb 5, 2026
CopilotAI changed the title Refactor evaluation samples: Replace instructional Console.WriteLine with real implementationsUncomment evaluation sample function definitions, keep only invocations commentedFeb 5, 2026
@markwallace-microsoftMark Wallace (markwallace-microsoft) added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs .NET Usage: [Issues, PRs], Target: .Net labels Feb 10, 2026
@github-actionsgithub-actionsBot changed the title Uncomment evaluation sample function definitions, keep only invocations commented.NET: Uncomment evaluation sample function definitions, keep only invocations commentedFeb 10, 2026
CopilotAIand others added 6 commits February 16, 2026 20:47
@rogerbarreto
Initial plan
bcb947c
@rogerbarreto
Add Foundry evaluation samples for Red Teaming and Self-Reflection
5948eff
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
@rogerbarreto
Refactor evaluation samples with real implementations in local functions
1dd32f2
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
@rogerbarreto
Uncomment function signatures and bodies, keep only invocations comme…
bfea890
…nted
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
@rogerbarreto
Update Foundry evaluation samples with observability support
14ffe13
@rogerbarreto
Restructure evaluation samples to follow FoundryAgents naming convention
af12ba1
- Rename Evaluation/Evaluation_StepXX to FoundryAgents_Evaluations_StepXX
- Add evaluation projects to slnx
- Fix var usage, apply dotnet format, use DefaultAzureCredential
- Add try/finally for agent cleanup
- Fix evaluator deployment name separation in Step02
- Update README references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rogerbarreto
Roger Barreto (rogerbarreto)force-pushed the copilot/support-foundry-observability branch from 343b0df to af12ba1CompareFebruary 16, 2026 20:56
@rogerbarretoRoger Barreto (rogerbarreto) changed the title .NET: Uncomment evaluation sample function definitions, keep only invocations commented.NET: Add Foundry Evaluation samples (Safety + Quality)Feb 16, 2026
@rogerbarreto
Roger Barreto (rogerbarreto) marked this pull request as ready for review February 16, 2026 20:59
CopilotAI review requested due to automatic review settings February 16, 2026 20:59
This was referenced Mar 9, 2026
Closed
Closed
Open
This was referenced Mar 16, 2026
Closed
Merged
Closed
Open
Closed
Open
Closed
Closed
Closed
This was referenced Mar 23, 2026
Merged
Closed
This was referenced Apr 6, 2026
Closed
Open
Open
@dependabotdependabotBot mentioned this pull request Apr 24, 2026
Merged
This was referenced May 2, 2026
Closed
Closed
This was referenced May 9, 2026
Closed
Closed
Closed
Closed
Closed
This was referenced May 18, 2026
Closed
Closed
@dependabotdependabotBot mentioned this pull request Jun 10, 2026
Closed
@dependabotdependabotBot mentioned this pull request Jun 24, 2026
Closed
@dependabotdependabotBot mentioned this pull request Jul 15, 2026
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@SergeyMenshykhSergeyMenshykhSergeyMenshykh approved these changes
@westey-mwestey (westey-m)westey-m approved these changes
@rogerbarretoRoger Barreto (rogerbarreto)Awaiting requested review from rogerbarretorogerbarreto is a code owner

Assignees

@rogerbarretoRoger Barreto (rogerbarreto)

Copilot code reviewCopilot

Labels

documentationUsage: [Issues, PRs], Target: documentation in the code base and learn docs.NETUsage: [Issues, PRs], Target: .Net

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

.NET: Support for Foundry Observability and Evaluations

6 participants

@SergeyMenshykh@westey-m@rogerbarreto@markwallace-microsoft