[BREAKING] Python: Consolidate OTel GenAI Semantic Conventions versions

[Tao Chen (TaoChenOSU)]

Motivation & Context

Addresses #7657 and #3603

MAF's observability currently emits a mixed of traces and events from different OTel GenAI semantic convention versions. We need to support both the v1.36.0 OTel GenAI Semantic Conventions as the baseline version and all versions above it as latest versions.

Description & Review Guide

• What are the major changes?
  • Introduce two new flags to control what OTel telemetry will be generate:
    • OTEL_SEMCONV_STABILITY_OPT_IN: control whether MAF will emit telemetry based on the baseline v1.36.0 convention or the latest experimental conventions.
    • ENABLE_MESSAGE_EVENTS: control if MAF will emit GenAI messages as events (defined by v1.36.0) while MAF is configured to use the experimental conventions. This is the most obvious difference between the stable version and the experimental ones.
    • The default for MAF is to emit using the experimental conventions and also emit GenAI messages as events, such that we don't break customers.
    • Attributes are added, removed, or renamed between stable and experimental. They are also included in this PR.
• What is the impact of these changes?
  • This PR introduces a clear contract between MAF and customers on what telemetry to expect.
  • This PR is considered breaking for customers who depend on the span attributes defined in v1.36.0 but not above such as gen_ai.system (replaced by gen_ai_provider.name in experimental) or message events that were previously incorrectly formatted (now corrected by this PR). These are considered acceptable as they are either very easy to migrate or was never a public contract.
• What do you want reviewers to focus on?
  • The documentation of the new flags.

Related Issue

Fixes#7657

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[Copilot]

Pull request overview

Adds configurable OpenTelemetry GenAI semantic-convention versioning to Python observability.

Changes:

• Adds stable/experimental convention and message-event settings.
• Gates provider, message, and tool-call telemetry attributes.
• Adds documentation and tests for version-specific behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
python/samples/02-agents/observability/README.md	Documents semantic-convention settings.
python/packages/core/tests/core/test_tools.py	Tests stable tool telemetry.
python/packages/core/tests/core/test_observability.py	Tests convention selection and message capture.
python/packages/core/tests/conftest.py	Resets new environment variables.
python/packages/core/agent_framework/observability.py	Implements convention-aware telemetry.
python/packages/core/agent_framework/_tools.py	Gates experimental tool attributes.
python/packages/core/agent_framework/_harness/_background_agents.py	Applies formatting-only changes.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 8c429189690c
Model:gpt-5.6-sol

Overview

The PR centralizes GenAI semantic-convention selection, preserves the existing experimental default, and gates sensitive tool payloads and message attributes with focused regression coverage. Provider-name switching and comma-separated opt-in parsing are well guarded. Two gaps remain in stable mode: output events do not satisfy the v1.36 choice-event contract, and spans can still contain attributes from newer conventions.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 1 file. Details are attached to the affected lines below.

Affected areas:python/packages/core/agent_framework/observability.py

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework
_sessions.py	1011	64	93%	164, 176–177, 207–208, 235–236, 270, 281, 295, 319, 364, 369, 371, 381, 413, 425, 435, 544, 613–614, 1355–1359, 1374, 1404, 1441–1442, 1456, 1458, 1478, 1480, 1579, 1619, 1696, 1700, 1710, 1927, 1960–1961, 1966, 1981, 2059–2060, 2062, 2146, 2245, 2318, 2333, 2338–2340, 2364, 2383, 2386, 2394–2395, 2407–2408, 2420, 2430, 2460
_tools.py	1531	98	93%	232–233, 410, 412, 425, 450–452, 460, 478, 492, 499, 506, 529, 531, 538, 546, 681, 715–717, 720–722, 724, 730, 785–787, 813, 839, 843, 881–883, 887, 1060, 1072, 1079–1082, 1103, 1111, 1125–1127, 1501, 1586, 1699–1700, 1757, 1804, 1811–1812, 1916, 1993, 2089, 2103, 2106, 2113, 2116, 2122, 2134, 2151, 2160, 2168, 2172, 2192, 2194, 2201, 2259, 2262, 2285, 2292, 2297–2298, 2301, 2305, 2308, 2330, 2364, 2432, 2461–2462, 2559, 2587, 2627, 2630, 2687, 2840, 2949, 3122, 3125, 3135, 3152–3153, 3621
observability.py	1133	84	92%	465, 467–468, 471, 474, 477–478, 483–484, 490–491, 497–498, 505, 507–508, 511, 514, 517–518, 523–524, 530–531, 537–538, 545, 722–723, 983, 987–989, 991, 999–1000, 1004, 1048, 1050, 1061–1063, 1065–1067, 1071, 1079, 1203–1204, 1448, 1450, 1452, 1763–1764, 2034, 2076–2077, 2264, 2513–2514, 2518, 2558–2559, 2563, 2604–2607, 2648–2649, 2822, 2825, 2837, 2854, 2858–2859, 2862, 2868, 2888, 2964, 2998, 3100, 3337, 3339
TOTAL	47336	4369	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9642	36 💤	0 ❌	0 🔥	2m 25s ⏱️

[github-actions]

MAF Automated Review — Iteration 2

Result: Findings reported
Scope: 4 net-new commit(s): 0f54a9ce27b0, 55886b0c2628, 3ae7e717d4ca, 46e73006380b
Model:gpt-5.6-sol

Overview

The review found 4 verified inline finding(s).

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
4 verified findings remained after source verification (4 medium) across 1 file. Details are attached to the affected lines below.

Affected areas:python/packages/core/agent_framework/observability.py