[METRICS] Fix stale async attribute sets in cumulative exports (#4108)

[vahle-at-psu]

Summary

Fixes #4108.

This PR continues the work from #4140 by @pranitaurlam. Since the original author has been unresponsive for several weeks and the fix is needed in production, I am taking over the PR, giving full credit to @pranitaurlam for the original analysis and fix skeleton.

Async instruments (ObservableCounter, ObservableGauge, ObservableUpDownCounter) under cumulative temporality emitted attribute sets indefinitely after the callback stopped reporting them, violating the OTel spec:

The implementation SHOULD NOT produce aggregated metric data for a
previously-observed attribute set which is not observed during a
successful callback.

---

Root cause

TemporalMetricStorage::buildMetrics() unconditionally carried every entry from last_reported_metrics_ into the cumulative output, even when the attribute set was absent from the current delta.

---

Changes

temporal_metric_storage.h / .cc

• Add is_async_ boolean (default false) to TemporalMetricStorage.
• In the cumulative merge lambda, add else if (!is_async_) guard: sync instruments still carry forward all attribute sets (existing behaviour, spec-correct); async instruments skip entries not present in the current delta.

async_metric_storage.h

• Pass is_async = true when constructing TemporalMetricStorage.
• Do not prune cumulative_hash_map_ in Collect() (deviation from [METRICS] Drop stale async attribute sets from cumulative exports #4140): the absolute-value baseline is preserved across absent cycles. This ensures correct incremental delta computation when an attribute set reappears after a gap (delta = new − last_seen, not new), consistent with opentelemetry-dotnet#6883. Stale entries are suppressed at export time by the is_async_ guard instead.

---

Behaviour

Scenario	Before	After
Async cumulative, attribute dropped by callback	Emitted forever with stale value	Dropped from next export
Async delta, attribute reappears after gap	Emitted as full new value	Emitted as increment since last seen
Sync cumulative, attribute not measured this cycle	Carried forward (correct)	Unchanged
Delta temporality (async or sync)	Correct (unaffected)	Unchanged

Sister SDKs with equivalent fixes:

• opentelemetry-dotnet#6883 (merged Feb 2026)
• opentelemetry-rust#2618 (merged Feb 2025)

---

Tests

Two regression tests added in async_metric_storage_test.cc, addressing lalitb's review request on #4140:

• StaleAttributeSetDroppedInCumulativeExport: attribute present in
  collection 1, absent in collection 2 → must not appear in collection
  2's cumulative export.
• AttributeReappearanceAfterGapDeltaTemporality: A=10 → missing → A=11 under delta temporality → emitted delta must be 1, not 11,
  confirming the baseline is preserved across the absent cycle.

---

Co-authored-by: pranitaurlam 227409059+pranitaurlam@users.noreply.github.com

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: pranitaurlam / name: pranitaurlam (e9cc6f3)
• ✅ login: vahle-at-psu / name: Patrick Summerer (e9cc6f3)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.53%. Comparing base (84fe660) to head (70a6832).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4484      +/-   ##
==========================================
+ Coverage   83.53%   83.53%   +0.01%     
==========================================
  Files         522      522              
  Lines       20447    20452       +5     
==========================================
+ Hits        17078    17083       +5     
  Misses       3369     3369              

Files with missing lines	Coverage Δ
...telemetry/sdk/metrics/state/async_metric_storage.h	93.19% <100.00%> (ø)
sdk/src/metrics/state/temporal_metric_storage.cc	100.00% <100.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mateenali66]

Built this on 731099b, the reported case is fixed. One problem with suppressing at export time.

merged_metrics is both the running cumulative state and the exported set (temporal_metric_storage.cc:179-190), so the new else if (!is_async_) at :163 does not just omit the point, it drops the baseline. On reappearance Record supplies delta = new - last_seen with nothing left to merge onto, so the cumulative point is the increment. A=10, absent, A=30, one collector:

cycle	main	this PR	expected
A=10	10	10	10
absent	10 (stale, #4108)	nothing	nothing
A=30	30	20	30

AttributeReappearanceAfterGapDeltaTemporality misses it because one collector under delta returns on the fast path at :67, before the changed branch. Happy to post the cumulative version of that test.

Would you rather keep the carry-forward in last_reported_metrics_ and filter when building the MetricData, so the baseline survives the gap?

[vahle-at-psu]

Built this on 731099b, the reported case is fixed. One problem with suppressing at export time.

merged_metrics is both the running cumulative state and the exported set (temporal_metric_storage.cc:179-190), so the new else if (!is_async_) at :163 does not just omit the point, it drops the baseline. On reappearance Record supplies delta = new - last_seen with nothing left to merge onto, so the cumulative point is the increment. A=10, absent, A=30, one collector:
cycle main this PR expected
A=10 10 10 10
absent 10 (stale, #4108) nothing nothing
A=30 30 20 30

AttributeReappearanceAfterGapDeltaTemporality misses it because one collector under delta returns on the fast path at :67, before the changed branch. Happy to post the cumulative version of that test.

Would you rather keep the carry-forward in last_reported_metrics_ and filter when building the MetricData, so the baseline survives the gap?

good catch!

I moved the is_async guard logic to the export callback, so the baseline state gets kept.
2 tests have been added:

• AttributeReappearanceAfterGapCumulativeTemporality
• MultiCollectorDeltaTemporality to test the slow path

Hope everything is addressed now :-)

[mateenali66]

rebuilt on 70a6832e. the baseline survives now. A=10, absent, A=30 on one collector gives 10, nothing, 30, and AttributeReappearanceAfterGapCumulativeTemporality pins exactly that. 226 metric tests pass.

also ran the two-collector cumulative case, since the new multi-collector test is delta and the is_async_ filter only fires on cumulative. second collector's interval spanning a cycle it did not itself collect in: c2 reports 10, exports zero points at the gap, then reports 30 on reappearance rather than 20 or 50. the carry-forward holds on the slow path too.

one note, not a blocker. the filter tests delta_metrics, which is the current call's map rather than the union of that collector's unreported deltas. with two collectors, c2 can skip an export for an attribute that another collector's cycle observed inside c2's own interval. the cumulative value is still right when it reappears, so this is a cadence difference rather than lost data, but worth confirming it is deliberate.

looks right to me otherwise.

[vahle-at-psu]

rebuilt on 70a6832e. the baseline survives now. A=10, absent, A=30 on one collector gives 10, nothing, 30, and AttributeReappearanceAfterGapCumulativeTemporality pins exactly that. 226 metric tests pass.

also ran the two-collector cumulative case, since the new multi-collector test is delta and the is_async_ filter only fires on cumulative. second collector's interval spanning a cycle it did not itself collect in: c2 reports 10, exports zero points at the gap, then reports 30 on reappearance rather than 20 or 50. the carry-forward holds on the slow path too.

one note, not a blocker. the filter tests delta_metrics, which is the current call's map rather than the union of that collector's unreported deltas. with two collectors, c2 can skip an export for an attribute that another collector's cycle observed inside c2's own interval. the cumulative value is still right when it reappears, so this is a cadence difference rather than lost data, but worth confirming it is deliberate.

looks right to me otherwise.

Again well analyzed - thank you!
I thought it would be better if I again change the introduced tests to run as fixture with both Delta and Cumulative temporality like the existing tests. Now they cover more cases inherently.
The cadence in the multi-collector case could be mitigated by remembering the individual collections and then filter on this set.