Fix OTel timer metrics using Gauge instead of Histogram

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865

[namratachaudhary]

Timer and timing metrics in the OTel logger were recorded using a Gauge instrument, which does not preserve multiple observations within an export interval. In practice, if the scheduler loop runs many times between exports, only a single value is retained, and the distribution of durations is lost.

This PR switches timing() and timer() to use a Histogram instrument, which captures duration distributions via count, sum, and bucketed observations. This enables accurate downstream analysis such as percentiles (p50, p95, p99).

The OTel supplementary guidelines recommend Histogram for measurements where "the statistics about this thing are likely to be meaningful" and durations are a canonical example.

Changes

• _OtelTimer.stop() and SafeOtelLogger.timing() now call record_histogram_value() instead of set_gauge_value()
• New InternalHistogram class wrapping meter.create_histogram(unit="ms")
• New MetricsMap.record_histogram_value() method
• get_otel_logger() configures the MeterProvider with a View that applies ExponentialBucketHistogramAggregation to all Histogram instruments — bucket boundaries adapt automatically to the observed data range without requiring hand-tuned explicit boundaries for metrics that span very different scales (ms to hours)
• Tests updated: test_timing_existing_metric now verifies both observations are recorded (record.call_count == 2), not just the last value; new test verifies the exponential histogram view is wired into the MeterProvider

Users affected

Affected metrics: all timer/timing metrics emitted via the OTel logger (e.g. scheduler.scheduler_loop_duration, dagrun.dependency-check, task.duration, etc.)

OTel users: the instrument type for timer/timing metrics changes from Gauge to Histogram. Dashboards or alerts treating these as Gauges may need to be updated.

Backend compatibility: exponential (native) histograms require Prometheus 2.40+ with native histogram scraping enabled and OTel Collector 0.70+. Deployments on older backends will have the collector downconvert to explicit buckets automatically — no data is lost, but the adaptive bucketing benefit is reduced. Both query styles work:

Before (Gauge):

airflow_task_duration{dag_id="my_dag", task_id="my_task"}

After (Histogram — explicit buckets, works on all backends):

# p95
histogram_quantile(0.95,
rate(airflow_task_duration_bucket{dag_id="my_dag", task_id="my_task"}[5m])
)
# Average duration
rate(airflow_task_duration_sum{dag_id="my_dag", task_id="my_task"}[5m])
/ rate(airflow_task_duration_count{dag_id="my_dag", task_id="my_task"}[5m])

After (Histogram — native histogram queries, Prometheus 2.40+):

histogram_quantile(0.95, rate(airflow_task_duration[5m]))

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[vatsrahul1001]

Manual backport up for review #66865