Skip to content

feat(runtime): Update pollrate behavior and add tests for it - #890

Open
CagriYonca wants to merge 2 commits into
mainfrom
pollrate-new-metrics
Open

feat(runtime): Update pollrate behavior and add tests for it#890
CagriYonca wants to merge 2 commits into
mainfrom
pollrate-new-metrics

Conversation

@CagriYonca

@CagriYoncaCagriYonca commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Why

The tracer previously capped poll_rate to a maximum of 5 seconds,
silently resetting any higher value back to 1. This prevented customers
from benefiting from coarser poll rates (e.g. 60 s, 120 s) to reduce
observability cost.

Additionally, GC metrics were collected using gc.get_count() and
gc.get_threshold(), which do not reflect actual GC activity. The correct
source is gc.get_stats(), which tracks cumulative collections,
collected, and uncollectable counters per generation — consistent with
the OpenTelemetry CPython semantic conventions
(cpython.gc.collections, cpython.gc.collected_objects,
cpython.gc.uncollectable_objects) and the
OpenTelemetry Python contrib system_metrics instrumentation,
both of which source these metrics from gc.get_stats().

Finally, the host agent timeout detection used a fixed 60-second window,
causing spurious disconnects when poll_rate exceeded 30 seconds.

What

  • options.py: Replaced MAX_POLL_RATE = 5 with a VALID_POLL_RATES
    list [1, 5, 10, 20, 30, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600].
    set_poll_rate() now rounds to the nearest valid value instead of
    rejecting anything outside {1, 5}.

  • runtime.py: Switched GC collection from gc.get_count() /
    gc.get_threshold() to gc.get_stats(). Reports per-generation deltas
    for collections, collected, and uncollectable (keys
    gc.collections0-2, gc.collected0-2, gc.uncollectable0-2).

  • host.py: Timeout window is now max(60, poll_rate * 2) seconds to
    avoid false resets at high poll rates. Added _send_heartbeat() — a
    lightweight HEAD request sent when no metrics/spans/profiles are
    produced in a cycle, keeping last_seen current between polls.

Testing

Unit tests are included for all changed behaviours in:
tests/test_options.py, tests/collector/helpers/test_collector_runtime.py,
tests/collector/test_host_collector.py, tests/agent/test_host.py

@CagriYoncaCagriYonca self-assigned this Aug 7, 2026
@CagriYonca
CagriYonca requested a review from a team as a code ownerAugust 7, 2026 10:40
@CagriYonca
CagriYonca marked this pull request as draft August 7, 2026 12:48
@CagriYonca
CagriYoncaforce-pushed the pollrate-new-metrics branch 2 times, most recently from 457617c to b7ce752CompareAugust 11, 2026 15:09
Signed-off-by: Cagri Yonca <cagri@ibm.com>
…esponding metrics.
Signed-off-by: Cagri Yonca <cagri@ibm.com>
@CagriYoncaCagriYonca changed the title feat(runtime): Add pauseMs and runCount metrics to runtimefeat(runtime): Update pollrate behavior and add tests for itAug 11, 2026
@CagriYonca
CagriYonca marked this pull request as ready for review August 11, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@CagriYonca