Skip to content

fix: lock CPU power history swap - #1335

Open
davidberenstein1957 wants to merge 1 commit into
masterfrom
fix/cpu-power-history-race
Open

fix: lock CPU power history swap#1335
davidberenstein1957 wants to merge 1 commit into
masterfrom
fix/cpu-power-history-race

Conversation

@davidberenstein1957

Copy link
Copy Markdown
Collaborator

Fixes a lost-update race on CPU._power_history.

What changed

CPU._power_history is appended to by the 1 Hz monitor scheduler thread (CPU.monitor_power) and drained by the measure_power_secs scheduler thread (CPU.total_power) — two distinct PeriodicScheduler timer threads (codecarbon/emissions_tracker.py:345-352). The read-then-rebind in total_power was not atomic, so any sample appended between the list comprehension and self._power_history = [] went into the discarded list.

  • Added a threading.Lock to CPU, held only for the O(1) list swap. _get_power_from_cpus() is called outside the lock in both paths, so a slow backend (e.g. the IntelPowerGadget subprocess) never blocks the monitor thread.
  • Removed the unreachable if not power_history_in_W: branch — a sample is unconditionally appended before the average, so the list is never empty.

Behaviour is otherwise unchanged, including the extra synchronous sample taken at measurement time.

Why

In cpu_load mode the reported cpu_power is the mean of the buffered samples. The samples dropped in the swap window are exactly those taken while a slow measurement is in flight, so the resulting error is small but systematic and silent.

How it was verified

New test tests/test_cpu_load.py::TestCPULoad::test_cpu_total_power_keeps_samples_added_while_draining makes the race deterministic (a list subclass that fires monitor_power() at iteration exhaustion, i.e. inside the lost-update window) and asserts the injected sample survives. It fails on master and passes with this change. uv run pytest tests/test_cpu_load.py -q → 9 passed.

Note: uv run task format reformats a large number of unrelated files on the current tree, so only the two touched files are included here.

Closes#1315

🤖 Generated with Claude Code

@codecov

codecovBot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.48%. Comparing base (3ec31a0) to head (f72f432).

Additional details and impacted files
@@ Coverage Diff @@## master #1335 +/- ##
==========================================
+ Coverage 91.43% 91.48% +0.04% 
==========================================
Files 49 49 Lines 5057 5060 +3 ==========================================
+ Hits 4624 4629 +5 + Misses 433 431 -2 

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 12, 2026 19:14
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code ownerAugust 12, 2026 19:14
@davidberenstein1957
davidberenstein1957force-pushed the fix/cpu-power-history-race branch 2 times, most recently from 00f2a1e to b813d76CompareAugust 19, 2026 14:29
The 1 Hz monitor scheduler thread appends to `CPU._power_history` while the
measurement scheduler thread drains it. The read-then-rebind in
`total_power()` is not atomic, so any sample appended between the list
comprehension and the rebinding was written to the discarded list and lost,
biasing the reported `cpu_power`.
Take the swap under a lock, held only for the O(1) rebinding so a slow
`_get_power_from_cpus()` backend never blocks the monitor thread. Also drop
the unreachable empty-history branch: a sample is always appended before the
average. The test asserts the drain invariant rather than the drain shape, so
it does not pin the implementation.
Closes#1315
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU power samples are silently dropped by a race in CPU.total_power()

1 participant

@davidberenstein1957