Add task.execute detail span around task execute callable by dstandish · Pull Request #67877 · apache/airflow

dstandish · 2026-06-01T22:45:02Z

Extracts the execute-callable invocation (including the execution-timeout wrapper) from _execute_task into a dedicated _run_execute_callable helper decorated with @detail_span("task.execute"). This emits a child span around the actual task execution when the task span detail level is greater than 1, giving finer-grained tracing of where time is spent within a task run.

Regular task failures mark the span as errored automatically via OpenTelemetry. AirflowTaskTimeout inherits from BaseException, which OpenTelemetry does not auto-record, so the timeout handler sets the span status to ERROR explicitly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

uranusjr

Pending Kaxil’s feedback

kaxil

LGTM -- clean extraction, and the task.execute span now correctly nests the operator's child spans (verified the nesting mechanism end to end). Approving.

Two non-blocking questions left inline: the safeguard-tracker test doesn't assert tracker non-leakage (one extra assertion would pin it), and a confirm-intent note on the copy_context() snapshot now being taken after the pre-execute hooks. Neither gates merge. The timeout span-status follow-up is tracked in #69146.

When task span detail level is greater than 1, the actual execute call was not separately traced, making it hard to see how much of a task's runtime was spent in the operator's own work versus the surrounding setup. Wrapping the execute call in its own span gives that finer-grained breakdown. The contextvars context the callable runs in is snapshotted inside the new helper, after the span is current, so spans the operator emits during execute nest under it rather than alongside it.

PR #67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated there and the canary build started failing.

apache#69236) PR apache#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated there and the canary build started failing. (cherry picked from commit 55cdf67) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like apache#67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds.

apache#69236) PR apache#67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated there and the canary build started failing. (cherry picked from commit 55cdf67) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

#69250) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds.

… code changes (#69250) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds. (cherry picked from commit 5298431) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

#69236) PR #67877 wrapped the operator execute callable in a new task.execute detail span, so operator-emitted spans now nest under it instead of directly under _execute_task. Integration tests do not run on regular PRs, so the expected span hierarchy in the OTel integration test was not updated there and the canary build started failing. (cherry picked from commit 55cdf67) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

…69359) When task span detail level is greater than 1, the actual execute call was not separately traced, making it hard to see how much of a task's runtime was spent in the operator's own work versus the surrounding setup. Wrapping the execute call in its own span gives that finer-grained breakdown. The contextvars context the callable runs in is snapshotted inside the new helper, after the span is current, so spans the operator emits during execute nest under it rather than alongside it. (cherry picked from commit b006a97) Co-authored-by: Daniel Standish <15932138+dstandish@users.noreply.github.com>

… code changes (#69250) (#69285) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds. (cherry picked from commit 5298431) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

…69359) When task span detail level is greater than 1, the actual execute call was not separately traced, making it hard to see how much of a task's runtime was spent in the operator's own work versus the surrounding setup. Wrapping the execute call in its own span gives that finer-grained breakdown. The contextvars context the callable runs in is snapshotted inside the new helper, after the span is current, so spans the operator emits during execute nest under it rather than alongside it. (cherry picked from commit b006a97) Co-authored-by: Daniel Standish <15932138+dstandish@users.noreply.github.com>

… code changes (#69250) (#69285) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds. (cherry picked from commit 5298431) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

…69359) When task span detail level is greater than 1, the actual execute call was not separately traced, making it hard to see how much of a task's runtime was spent in the operator's own work versus the surrounding setup. Wrapping the execute call in its own span gives that finer-grained breakdown. The contextvars context the callable runs in is snapshotted inside the new helper, after the span is current, so spans the operator emits during execute nest under it rather than alongside it. (cherry picked from commit b006a97) Co-authored-by: Daniel Standish <15932138+dstandish@users.noreply.github.com>

… code changes (#69250) (#69285) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds. (cherry picked from commit 5298431) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

…69359) When task span detail level is greater than 1, the actual execute call was not separately traced, making it hard to see how much of a task's runtime was spent in the operator's own work versus the surrounding setup. Wrapping the execute call in its own span gives that finer-grained breakdown. The contextvars context the callable runs in is snapshotted inside the new helper, after the span is current, so spans the operator emits during execute nest under it rather than alongside it. (cherry picked from commit b006a97) Co-authored-by: Daniel Standish <15932138+dstandish@users.noreply.github.com>

… code changes (#69250) (#69285) The otel core integration was only triggered by observability sources, so PRs changing the spans the task runner emits (like #67877) or the otel integration tests themselves passed CI without running the tests that assert the span hierarchy, and breakage surfaced only in canary builds. (cherry picked from commit 5298431) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

dstandish · 2026-06-01T22:45:02Z

Extracts the execute-callable invocation (including the execution-timeout wrapper) from _execute_task into a dedicated _run_execute_callable helper decorated with @detail_span("task.execute"). This emits a child span around the actual task execution when the task span detail level is greater than 1, giving finer-grained tracing of where time is spent within a task run.

Regular task failures mark the span as errored automatically via OpenTelemetry. AirflowTaskTimeout inherits from BaseException, which OpenTelemetry does not auto-record, so the timeout handler sets the span status to ERROR explicitly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

uranusjr

Pending Kaxil’s feedback

kaxil