Skip to content

Python: Include arguments for zero-argument Foundry eval tool calls - #7724

Open
Luis Rodriguez (luisangelrod) wants to merge 1 commit into
microsoft:mainfrom
luisangelrod:fix/foundry-evals-zero-arg-tools
Open

Python: Include arguments for zero-argument Foundry eval tool calls#7724
Luis Rodriguez (luisangelrod) wants to merge 1 commit into
microsoft:mainfrom
luisangelrod:fix/foundry-evals-zero-arg-tools

Conversation

@luisangelrod

Copy link
Copy Markdown
Contributor

Motivation & Context

Foundry evaluation conversion currently omits the arguments field when a function call has None or an empty object for its arguments. This produces an incomplete tool-call record for valid zero-argument tools. The converter should emit an empty object so the evaluation payload remains schema-consistent.

Description & Review Guide

  • What are the major changes? The converter now always includes arguments, using an empty object when the source value is None. Regression tests cover both None and an explicitly empty argument object.
  • What is the impact of these changes? Zero-argument tool calls can be evaluated by Foundry without losing the required argument shape. Calls with populated arguments are unchanged.
  • What do you want reviewers to focus on? Whether normalizing missing arguments to an empty object is the expected representation for Foundry evaluation tool calls.

Related Issue

Fixes#7714

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds coverage and adjusts the eval conversion output so tool calls always include an arguments object, even when the tool call has zero/empty arguments.

Changes:

  • Add tests for assistant tool calls with arguments=None and arguments={}.
  • Update AgentEvalConverter.convert_message to always emit an arguments field, defaulting to {} when missing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

FileDescription
python/packages/foundry/tests/test_foundry_evals.pyAdds regression tests to ensure tool-call arguments are normalized to {} for zero/empty args.
python/packages/core/agent_framework/_evaluation.pyNormalizes tool-call conversion output to always include arguments (defaults to {} when None).

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pythonUsage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: FoundryEvals omits arguments for zero-argument tool calls, causing tool-aware Foundry evaluators to fail

2 participants

@luisangelrod