Fix OTel integration test after task.execute span addition

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)

[jason810496]

• related: Add task.execute detail span around task execute callable #67877
• fix upstream scheduled CI failure (https://github.com/apache/airflow/actions/runs/28562105904), I also encounter the same error in Clarify logging_config_class contract and document REMOTE_TASK_LOG #67104.

Why

#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated and the canary build started failing on test_dag_execution_succeeds[detail_spans].

What

• Add task.execute (child of _execute_task) to the expected span hierarchy of the detail_spans case.
• Re-parent sub_span1 from _execute_task to task.execute, matching what the runtime actually emits.

Verified with breeze testing core-integration-tests --integration otel --backend postgres: all 5 OTel integration tests pass.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Fable 5 following the guidelines

[Dev-iL]

My agent also suggested to harden the test harness:

The parametrized default_spans case (level None) doesn't have this problem because it uses a shallow span-detail level where task.execute isn't emitted at all — so a future span addition at deeper levels would again silently desync this hardcoded dict. Two low-cost improvements worth considering in a follow-up PR (not doing them now, since they're beyond fixing the immediate regression):

1. Assert on the full set of span names, not just the parent-map, and fail with a clear message like "new span X appeared — update expected_hierarchy" rather than a raw dict diff, so the next span-adding PR gets a legible pointer to this test.
2. Cross-reference TASK_SPAN_DETAIL_LEVEL_KEY additions: add a comment near @detail_span(...) call sites in task_runner.py pointing back at test_otel.py's detail_spans case, similar to how the tracking-issue link on line 458-459 already works — so PR authors adding a new @detail_span know to update this test.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.1 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[jason810496]

My agent also suggested to harden the test harness:

This make sense to me as well, but I will merge this now to make main green, please feel free to raise follow-up for the test hardening.

I will fix from the selective check side first, that we should trigger the otel integration test whenever the task-sdk task_runner.py change. This is the root cause of the main CI regression (the otel test never trigger at all for the #67877 !)