🌱 Add bounded metrics for device-tracker scan cycles - #23350
🌱 Add bounded metrics for device-tracker scan cycles#23350kubestellar-hive[bot] wants to merge 1 commit into
Conversation
The device-tracker background loop (pkg/agent/workers) runs a 60s scan
across all managed clusters entirely outside the HTTP request path, and
had no observability: no cycle counter, no duration histogram, and
ListClusters/GetNodes failures were only ever visible as a single
rate-limited slog line.
Add kc_device_tracker_scan_cycles_total, kc_device_tracker_scan_duration_seconds,
and kc_device_tracker_scan_errors_total{stage}, following the same bounded,
pull-only pattern already used for the stellar/gpu sweep loops in
pkg/api/metrics/metrics.go. stage is a fixed two-value set
(list_clusters, get_nodes) — no cluster or node identifiers are used as
labels. Metrics are exposed via the agent's existing /metrics endpoint;
no exporter or new external data flow is introduced.
Fixes #23349
Signed-off-by: kubestellar-hive[bot] <223556219+Copilot@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
✅ Deploy Preview for kubestellarconsole canceled.
|
|
👋 Hey @kubestellar-hive[bot] — thanks for opening this PR!
This is an automated message. |
|
Important Held for human sign-off on the direction, not on the code. This PR's only tracked rationale is #23349, which the hive filed itself — issue #23349 was filed by kubestellar-hive[bot] and no human has acknowledged it. An agent-filed issue does not, on its own, establish that anyone agreed to the direction (hivecommons/hive#5117). The change may well be right; nothing here is a review of it. To release the hold, acknowledge the direction on that issue — comment on it, assign yourself, or add the |
Fixes #23349
What
DeviceTracker.scanDevices(pkg/agent/workers/workers_device_tracker.go) runs a 60s scan loop across all managed clusters entirely outside the HTTP request path, and had no observability: no cycle counter, no duration histogram, andListClusters/GetNodesfailures were only ever visible as a single rate-limitedslog.Infoline.Change
Adds, in
pkg/agent/workers/device_tracker_metrics.go, following the existing bounded self-metrics pattern (seepkg/api/metrics/metrics.go's stellar/gpu sweep counters):kc_device_tracker_scan_cycles_total(counter)kc_device_tracker_scan_duration_seconds(histogram)kc_device_tracker_scan_errors_total{stage}(counter;stage∈{list_clusters, get_nodes}— fixed, bounded set, never a cluster/node identifier)Registered alongside the existing prediction metrics in
InitPredictionMetrics()and exposed via the agent's existing/metricsendpoint. No exporter or new external data flow is introduced — this is pull-only self-observability.Testing
go build ./pkg/agent/...go test ./pkg/agent/workers/...— passesgofmt/go vetclean— hive: agent=telemetry backend=copilot model=claude-sonnet-4-6