Consume the accumulated sub-threshold bias-shift distribution to calibrate the silent-drift threshold empirically
Source
Reader comment 3dn2b by reidmarlow on "A guard that has never fired and a guard that stopped running look identical on disk" (https://dev.to/pm25coder/a-guard-that-has-never-fired-and-a-guard-that-stopped-running-look-identical-on-disk-177f), 2026-08-29T23:07Z.
Reader's insight
"Logging the sub-threshold distribution turns an alert threshold from an arbitrary constant into an empirical boundary." Boolean monitors suffer survivor bias — you only know where the line should be after a false positive wakes someone up or a real breach sails past. The #1073 heartbeat made the sub-threshold spread visible (a histogram we own); the next step is to actually consume it.
Current state (master b0c67b7)
Proposed change
- Count the sub-threshold observations: append a
bias_shift_observed event (session, bias_shift, old_bias, new_bias, threshold) to the existing ~/.emrg/logs/usage-anchor.jsonl via the same _append_usage_anchor_event writer used by the drift event — sub-threshold values become countable, not just greppable. Keep the debug heartbeat line as-is. - Calibration readout script: a small script (precedent:
scripts/reader_fix_latency.py) that reads the accumulated bias_shift_observed distribution and prints n, median, p90/p99, max, and an empirical threshold suggestion (e.g. max observed × safety factor, or p99 × k) alongside the current 0.25 constant — so 25% becomes "a number someone can argue with".
This keeps the "log unconditionally, alert conditionally" principle and does not auto-tune the threshold silently — script-assisted decision first, auto-adjust (if ever) only later with explicit bounds.
Acceptance (self-verifiable)
- Sub-threshold
bias_shift values land in a countable store (JSONL event), not only a debug log line. - The calibration script prints
n + distribution summary + a suggested threshold from real event files. - Tests assert the new event shape and the script's output on a fixture distribution.
Consume the accumulated sub-threshold bias-shift distribution to calibrate the silent-drift threshold empirically
Source
Reader comment 3dn2b by reidmarlow on "A guard that has never fired and a guard that stopped running look identical on disk" (https://dev.to/pm25coder/a-guard-that-has-never-fired-and-a-guard-that-stopped-running-look-identical-on-disk-177f), 2026-08-29T23:07Z.
Reader's insight
"Logging the sub-threshold distribution turns an alert threshold from an arbitrary constant into an empirical boundary." Boolean monitors suffer survivor bias — you only know where the line should be after a false positive wakes someone up or a real breach sails past. The #1073 heartbeat made the sub-threshold spread visible (a histogram we own); the next step is to actually consume it.
Current state (master b0c67b7)
_detect_silent_anchor_drift(emrg/server/daemon.py:3094) computes the bias shift on every anchored round.anchor-bias-heartbeatdebug line logs every computed shift unconditionally (daemon.py:3128-3135, issue Make silent-drift 'never fired' measurable: unconditional bias_shift log + scheduled planted-fire test #1072 / PR emrg: usage anchor — unconditional bias-shift heartbeat + planted-fire assertions (issue #1072) #1073)._SILENT_DRIFT_THRESHOLD = 0.25(daemon.py:4300) remains an arbitrary constant.anchor_provider_driftevent appends to~/.emrg/logs/usage-anchor.jsonlonly when shift >= threshold (daemon.py:3136-3145).Proposed change
bias_shift_observedevent (session,bias_shift,old_bias,new_bias,threshold) to the existing~/.emrg/logs/usage-anchor.jsonlvia the same_append_usage_anchor_eventwriter used by the drift event — sub-threshold values become countable, not just greppable. Keep the debug heartbeat line as-is.scripts/reader_fix_latency.py) that reads the accumulatedbias_shift_observeddistribution and printsn, median, p90/p99, max, and an empirical threshold suggestion (e.g. max observed × safety factor, or p99 × k) alongside the current 0.25 constant — so 25% becomes "a number someone can argue with".This keeps the "log unconditionally, alert conditionally" principle and does not auto-tune the threshold silently — script-assisted decision first, auto-adjust (if ever) only later with explicit bounds.
Acceptance (self-verifiable)
bias_shiftvalues land in a countable store (JSONL event), not only a debug log line.n+ distribution summary + a suggested threshold from real event files.