feat(agent-health): observer frames, agent-health.db, Health tab and buzz agents health (T17) - #37
Merged
Merged
Conversation
…T18) with T16 fixture tests Three design specs after Devin's 2026-09-06 decisions (pause on session limit, no seat rotation; only never-started batches replay; nothing discarded) and GPT-5.6 Sol's audit of the question set. Wave 6 tickets added to the implementation plan. crates/buzz-acp/src/reliability.rs holds the T16 fixture tests on the real log lines of 2026-09-02/03 plus the smallest stubs that let them compile. All seven are #[ignore] and fail with --ignored; T16 is ready when they pass un-ignored. Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
The harness used to dead-letter a batch after ten retries: it logged at ERROR, posted one warning and dropped the events. A Claude session limit lasts hours, so every message in that window died after 25 minutes. Nothing is discarded any more. A failure that used to dead-letter now parks the batch in an owner-only park file in the agent's state directory, and either replays it automatically after a successful live turn or hands it to the operator for review. - reliability/error_class.rs classifies a provider error and parses "resets 4:20am (America/Los_Angeles)" into the next occurrence of that wall time, via chrono-tz. An unparseable reset pauses 30 minutes; a pause is clamped at 6 hours. - reliability/state.rs holds the per-agent pause and the per-scope breakers: three consecutive provider errors open a breaker with a 10-minute probe and a 6-hour cap, then the batch parks. - reliability/state_dir.rs resolves BUZZ_ACP_STATE_DIR (0700 dir, 0600 files), falling back to ~/.buzz/.state/<pubkey prefix>/. - reliability/ledger.rs is the append-only ledger.jsonl: one serde struct per record kind, fsync per append, 30-day retention truncated on start and every 6 hours, 10 MB cap. - reliability/park.rs is parked.jsonl, written atomically, with caps on bytes, batches per scope and batches in total. - reliability/runtime.rs orders the writes: park before the batch is dropped, batch_replayed before the prompt is staged. - queue.rs: FlushBatch carries a stable batch_id; requeue no longer returns a batch to be discarded but hands it to the park path; stage_replay merges parked events ahead of newer ones with the "Delivered late" annotated section, reusing the cancelled-events merge and never editing the event text. - lib.rs gates dispatch on the pause and the breakers, drives the machinery on every prompt result, and accepts replay_batch, discard_batch, resume_now and keep_paused on the same owner-checked path as switch_model. The seven T16 fixture tests pass with their bodies unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Z6iidtozXxgx58BUZUKnu Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…nd in progress; review before use) Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…T17 step 1) ReliabilityRuntime now carries an optional ObserverHandle (with_observer) and, after a successful ledger append, emits a live observer frame for the nine health-relevant record kinds (batch_parked, batch_replayed, batch_needs_review, agent_paused, agent_resumed, breaker_opened, breaker_closed, relay_reconnected, and turn_finished errors remapped to turn_failed). This lets the desktop's future Health tab and buzz agents health track agent reliability from live frames without polling the ledger file or exposing raw provider error text: turn_failed frames carry only the error class, never the raw field. LedgerBody/LedgerRecord gained a channel_id() accessor so the frame's ObserverContext can carry the right channel. Wired into tokio_main's ReliabilityRuntime::open via with_observer(observer.clone()). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…nd state-dir fixtures (T16) Updates the two dead-letter tests to the park behaviour the design specifies, clears two clippy errors, adds the design's replay fixtures (#5, #6), and adds the desktop state-dir reserved-key and pubkey validation tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Renamed the private read_records helper to a public read_ledger_file so the desktop and CLI can read a live ledger.jsonl without going through Ledger::open, which truncates and rewrites the file on open. The function's behavior is unchanged (same byte and line caps, malformed lines skipped and counted, missing file returns empty); only its visibility and name changed, plus a doc comment stating the read-only contract. Added a regression test that writes a mix of good and malformed lines, reads them back through read_ledger_file, and asserts the file's bytes are byte-for-byte unchanged afterward, so any future change that reintroduces a rewrite side effect fails this test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
… (T17 step 3) Adds desktop/src-tauri/src/agent_health.rs, modelled on observed_unread.rs: a rebuildable SQLite agent-health.db (schema_meta + health_events, indexed on (agent,at) and (kind,at)), insert_event using INSERT OR IGNORE keyed on (agent, event_key) so re-syncing the ledger or re-delivering a frame never double-counts, and prune deleting rows older than the 30-day retention window (run once on every open_db). The dedupe key is agent|kind|batch_id-or-scope over the ledger's own at timestamp rather than the design's (agent, seq), because the observer's seq is process-local and restarts at 1 on every relaunch (R6) so it cannot dedupe across restarts or against a later ledger sync. Registers the new module in lib.rs beside observed_unread. Both TDD tests (insert_ignores_duplicate_event_key, retention_prunes_older_than_30_days) were written first against stub bodies, confirmed red, then made green by the real INSERT OR IGNORE / DELETE implementations; cargo clippy on the tauri crate is clean with -D warnings. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…ep 4) Add agent_health::sync_ledger, which reads an agent's ledger.jsonl through the T17-step-2 read-only reader and inserts each record into the health_events table via insert-or-ignore, so re-running it after a restart or a repeated call never double-counts a record. turn_finished with an error outcome is stored as kind turn_failed with raw dropped, matching the harness's own observer-frame mirror from step 1; every other outcome, plus turn_started, keeps its ledger kind so the desktop can later count turns over 24h/7d. Moved the buzz_acp_pkg dependency from dev-dependencies to dependencies in desktop/src-tauri/Cargo.toml: the plan's step 4 has agent_health.rs (production code, not test-only) call into buzz-acp's reliability::ledger module, which a dev-dependency cannot supply outside #[cfg(test)]. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…ation (T17 step 5) Add query_agent_health_summary (per-agent group-by over health_events: turns, failed, parked, needs_review, reconnects counts, last failure class/at, latest agent_paused.until, and open-breaker-without-close detection) and query_agent_health_events (kind/window filtering, descending order, hard cap at 200). Wire both behind the get_agent_health_summary and get_agent_health_events Tauri commands alongside the existing sync_agent_health and ingest_agent_health_frame, all managed by AgentHealthStore and registered in generate_handler!, so the Health tab and buzz agents health can read real per-agent counters and event history instead of empty stubs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…p 6) Add get_parked_batches (desktop Tauri command) reading the agent's park file via ParkFile::open/batches, mapping each ParkedBatch to a ParkedBatchView with a 120-char excerpt (ParkedEvent::excerpt) and no full message text. Add TS control senders (replayParkedBatch, discardParkedBatch, resumeAgentNow, keepAgentPaused) over the existing observer control channel, and extend the control payload type union and ParkedBatchView type in shared/api/types.ts to match the harness's replay_batch/discard_batch/resume_now/keep_paused control frames. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…(T16) state.rs: release probe permits on hold/spawn-failure, not just success — test_dispatch_pending_does_not_leak_probe_permit_on_hold state.rs: on_failure clears consecutive streak on Auth/CapacityExhausted too — non_provider_failure_resets_consecutive_streak state.rs: consecutive map entry removed on every terminal park/pause, bounded — test_consecutive_map_stays_bounded_across_many_scopes park.rs: reject an oversized individual serialized line before park commits — test_park_rejects_oversized_individual_line park.rs: scope-cap demotion computed before the single atomic commit — test_park_101st_batch_scope_cap_single_atomic_write state_dir.rs: write_atomic propagates parent-dir open/fsync errors — test_write_atomic_propagates_parent_dir_fsync_error ledger.rs: Ledger::open quarantines a dangling no-newline final line — test_ledger_open_quarantines_dangling_final_line_without_newline runtime.rs: discard() returns false when the ledger append failed — test_discard_fails_contract_when_ledger_append_fails lib.rs: handle_prompt_result branches on Disposition so a failed park always re-enters queue/hand-off — test_park_failure_does_not_discard_batch_on_hard_timeout_or_auth runtime.rs: plan_replay builds plans from whole batches under MAX_BATCH_EVENTS, leaving the rest parked — test_replay_plan_respects_max_batch_events_and_preserves_unincluded_batches lib.rs: Pause/OpenBreaker batches are durably parked, recoverable across restart — test_pause_held_batch_is_durable_across_restart lib.rs: state-dir open failure gets a bounded periodic reopen retry — test_state_dir_failure_refuses_work_and_picks_up_on_reopen lib.rs: owned select! timer arm wakes on min(pause.until, breaker.next_probe) — test_probe_timer_fires_without_external_relay_event lib.rs: dispatch_pending short-circuits on global pause before the per-scope loop — test_dispatch_pending_short_circuits_global_pause_in_o1 acp.rs/lib.rs: turn_saw_output mirrored to a shared atomic so a panic preserves it for parking — test_panicked_agent_after_output_parks_with_started_true_and_needs_review acp.rs: turn_saw_output set only on agent_message_chunk/tool_call, not every session/update — wire_session_info_and_available_commands_do_not_set_turn_saw_output, wire_agent_message_chunk_and_tool_call_set_turn_saw_output pool.rs: post_failure_notice retries with backoff and reports success via NoticeAck — test_failure_notice_not_consumed_until_ack_received queue.rs: mark_complete_preserving_retries keeps the retry count across Pause/BreakerOpen — test_retry_counts_preserved_across_pause_and_breaker error_class.rs/lib.rs: sanitize_error_diagnostic redacts/caps raw provider errors at the ACP boundary — test_error_boundary_sanitizes_diagnostic_and_preserves_raw_in_ledger desktop runtime.rs: apply_state_dir_env wired into the real spawn seam, test asserts via get_envs() (no ambient-env subprocess leak) — test_command_execution_overrides_and_ignores_ambient_state_dir Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Add agentHealthFrames.ts to parse the nine health-frame kinds off the live observer stream, wire observerRelayStore to ingest them via a new ingestAgentHealthFrame Tauri wrapper, and trigger a background health-store sync (agent_health::sync_for_agent) after managed-agent start/stop and once after launch restore, so the health store stays current without waiting for a manual refresh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Add buildHealthRows/hasHealthAlertBadge to reduce raw 24h/7d health counters into per-agent UI rows (state, pause window, turn/failure counts, last error), and add TanStack Query hooks (useAgentHealthSummaryQuery, useAgentHealthEventsQuery, useParkedBatchesQuery) that sync the local ledger then read the summary/events/parked-batches Tauri commands, refetching on app focus and on managed-agent-runtime-status events. This is the pure-reducer + data layer the Health tab (step 9) renders against. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Add a Health tab to the Agents screen alongside the existing Agents tab, driven by useHistorySearchState so the active tab round-trips through the URL. AgentHealthTab renders one row per agent (state badge, turn/failure counts, parked count, last error, reconnects) sourced from the Step 8 summary reducer, with remote-owned agents marked 'frames only, no local ledger'. Clicking a row opens AgentHealthDrawer (a Sheet), which shows the pause card (Resume now / Keep paused +1h), the failed/parked batch list with Retry and Discard actions wired to the Step 6 control senders, and the last 50 health events. Fixed one accessibility defect found during verification: the agent-name button and the row's Details button both carried the identical aria-label, violating AGENTS.md Review-Proven Rule 7 (one owner per actionable label); the name is now non-interactive text so the Details button is the row's single actionable control. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
… (T17 step 10)
Add agent_health_alerts::evaluate(), implementing the five design-`4 alert rules (parked batch >15 min, needs_review, breaker_opened, pause >1h, non-zero exit) with a once-per-(agent,rule)-per-hour rate limit backed by a new alert_state table in agent-health.db. sync_agent_health and ingest_agent_health_frame now return {inserted, alerts}, and the TS callers (syncAgentHealth, ingestAgentHealthFrame) deliver each alert via sendDesktopNotification.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz
Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Second and final fix round on the T16 reliability harness. Verified all 16 BLOCK findings in the delta-1 adversarial review (7 new regressions introduced by the first fix round, plus 9 sub-findings across 8 prior findings the first round left unresolved) and fixed every one: - park hand-off overflow no longer drops a batch (return_unparked hands it back instead of taking it by value) - the probe-timer busy-spin: dispatch now runs on every valid probe wake regardless of live queue depth, and pause/breaker deadlines reschedule forward instead of re-triggering on a stuck past deadline - cancelled_events (interrupted/replay carryover) are now persisted when a batch is parked instead of requeued - commit_replay rolls back earlier marks when a later one in the same plan fails; finish_replay retains in-flight ownership on partial removal failure - a directory-fsync failure after a successful atomic rename no longer makes write_atomic report the write as lost - the park reader quarantines corrupt/over-cap records to a .corrupt sibling file instead of silently truncating and admitting them - open breakers are now bounded (MAX_OPEN_BREAKERS) and swept for 6h expiry independent of new traffic on the scope - push refuses new admission at cap while reliability state is unavailable, instead of evicting an already-queued message with nowhere to land - pause/breaker probe leases are released on every dispatched outcome (retry, park, panic), not only the hold paths - a panicked turn that already produced output is parked directly as needs_review instead of losing its started flag through the plain retry queue - the ledger sanitizes a dangling partial line before every append, not only at open - discard_batch distinguishes NotFound from Discarded from DiscardedUnrecorded instead of collapsing the last two into "unknown_batch" - a ledger write failure during park_batch now surfaces a channel notice (the previously dead-code state_write_failures function) - failure-notice retries extended from ~15s to ~7-8 minutes of backoff - credential redaction now normalizes key names (strips separators/casing) and covers common token prefixes (ghp_, gho_, etc.), closing JSON/env-var/ hyphenated-key gaps - the desktop state-dir env application is now gated by a StateDirApplied proof token consumed at the real spawn call, matching the EffortApplied/McpEnvApplied pattern, so the production seam cannot be deleted or reordered without a compile error Two items are disclosed as scoped follow-ups rather than force-fit into this round: a fully durable cross-restart notice outbox, and extending the reliability-unavailable admission gate to the aggregate per-channel cap (the per-scope gate already covers the finding's exact repro). Verified: cargo fmt/clippy clean for buzz-acp and desktop/src-tauri; targeted test suites (reliability, queue, hard_timeout, managed_agents) all green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Renders a red '!' SidebarMenuBadge on the Agents entry in the pinned sidebar header when hasHealthAlertBadge (Step 8's reducer over the health summary query) reports a needs-review batch or an open breaker for any agent, so the operator sees trouble without opening the Agents screen. Threaded agentsHealthAlert through AppShell -> AppSidebar -> AppSidebarPinnedHeader as a plain boolean prop, mirroring the existing homeBadgeCount badge pattern in the same component. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…ep 12) Seeds the e2e mock bridge with an agentHealth option (summary, events, parked) and wires the five agent-health Tauri commands (get_agent_health_summary, get_agent_health_events, sync_agent_health, ingest_agent_health_frame, get_parked_batches) next to list_managed_agent_runtimes; widens the observer control payload type from a union to a string so replay_batch/discard_batch/etc pass through the existing __BUZZ_E2E_OBSERVER_CONTROLS__ capture. Adds an agent-health fixture (one paused agent, one agent with a needs-review parked batch) and a Playwright smoke spec that opens the Health tab, checks the paused/needs-review rows, opens the drawer, clicks Retry, and asserts the replay_batch control fires and the sidebar health badge shows. Registers the spec in the smoke project's testMatch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Add a pure summarize() reducer in buzz-acp/reliability/health.rs that scans ledger records for an agent and produces windowed counters (turns, failed, parked, needs_review, reconnects, last error) plus current state (active/paused/breaker/offline), then wire buzz agents health [--since 24h|7d] [--json] in buzz-cli to read local ledger.jsonl files directly (via the Step 2 read_ledger_file reader) and print the same table with no relay connection required, following the Pack local-only early-return precedent. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…bility PR #32 landed on zs/main while this branch was in flight and touched the same two files this branch's T16 implementation owns. Resolved by keeping this branch's implementation in both cases, since #32's content was the earlier design-stage scaffolding for the same work: - crates/buzz-acp/src/reliability.rs: add/add conflict. #32 added a stub module (ErrorClass, Action, a no-op ReliabilityState, and #[ignore]'d fixture tests). This branch already has the full T16 implementation (error_class, state, state_dir, ledger, park, notices, runtime submodules) whose own test suite includes every one of #32's fixture cases unignored and passing, plus additional coverage. Kept this branch's file as-is (`git checkout --ours`); nothing from #32's stub needed folding in. - crates/buzz-acp/src/lib.rs: content conflict on the module declaration line — #32 added `mod reliability;` (private, sufficient for its internal stub) where this branch already had `pub mod reliability;` (public, since the real runtime's types are consumed elsewhere in this file and are intended for future external test crates). Kept this branch's `pub mod reliability;`. All other files auto-merged cleanly; PR #32's three unrelated design docs (docs/plans/2026-09-04-zs-implementation-plan.md, 2026-09-06-agent-health-design.md, 2026-09-06-agent-self-improvement-design.md) already exist identically on this branch, so the merge is a no-op outside of history. # Conflicts: # crates/buzz-acp/src/lib.rs # crates/buzz-acp/src/reliability.rs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
PR #35's Windows clippy job failed with 'unused import: super::*' at state_dir.rs:192, because the sole test in that module is cfg(unix) but the use super::* sat under a bare cfg(test), so on non-unix targets the module still compiles with the glob import but no unix-gated test left to use it, tripping -D warnings. Moved the unix gate up to the module attribute (cfg(all(test, unix))) so the whole test module — import included — compiles only on unix, where it behaves exactly as before; on Windows the module is skipped entirely instead of leaving an unused import. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
1. file-size gate failing on agent_health.rs -> split #[cfg(test)] mod into agent_health/tests.rs -> check-file-sizes.mjs exits 0
2. events query materialized all rows before capping -> pushed kind filter + LIMIT into SQL -> events_are_filtered_by_kind_and_capped (10k-row wall-time bound)
3. summary query loaded entire matching history -> aggregated counters in SQL, split windowed counters from an unwindowed current-state scan -> summary_counts_per_agent_within_window
4. health DB had no row/byte ceiling -> prune() enforces a 1000-row / 10MiB budget, oldest rows first -> prune_enforces_row_and_byte_budget_oldest_first
5. frame timestamps could evade retention -> reject a frame timestamp more than 5 minutes in the future -> future_timestamp_frame_is_rejected
6. Rust accepted arbitrary health-frame content -> HealthFrame gets deny_unknown_fields, a kind allow-list, and length caps -> ingest_rejects_unknown_kind_oversized_class_and_invalid_agent
7. equivalent timestamps defeated dedup (+00:00 vs Z) -> canonicalize the timestamp before hashing the event key -> duplicate_events_with_different_rfc3339_timezone_notations_deduplicate
8. resolved needs-review batches kept driving the badge -> track active_needs_review as a set, cleared on batch_replayed/discarded -> needs_review_cleared_by_replayed_or_discarded
9. agent_resumed did not clear pause -> added an agent_resumed match arm that clears latest_paused_until -> agent_paused_cleared_by_agent_resumed
10. breaker state collapsed independent scopes into one bool -> per-scope open-set tracking in both the desktop reducer and buzz-acp's CLI reducer -> breaker_open_persists_outside_counters_window, breaker_tracks_per_scope
11. current pause/breaker state was limited to the requested activity window -> added an unwindowed 30-day state scan merged into the summary -> breaker_open_persists_outside_counters_window
12. unvalidated since_hours could overflow -> checked-arithmetic range validation (0..=720h) before computing cutoff -> invalid_since_hours_is_rejected
13. sync_agent_health converted failures into success -> extracted resolve_managed_agents_for_sync: hard error on a broken store for a full sync, per-agent recorded error otherwise -> resolve_managed_agents_for_sync_{errors_hard_on_full_sync,degrades_on_targeted_sync,passes_through_on_success}
14. ingest swallowed park-file failures -> read_parked_batches now distinguishes a missing dir/file from corrupt content and propagates the latter as Err -> read_parked_batches_distinguishes_missing_from_corrupt
15. frontend hid sync failure and showed stale data as current -> fetchAgentHealthSummary surfaces syncError, AgentHealthTab renders a stale-data banner -> fetchAgentHealthSummaryDegradesOnSyncRejection
16. live frame ingestion had no bounded queue/retry -> bounded serial per-agent queue (cap 50) with one retry on failure -> observerRelayHealthQueue.test.mjs
17. detached ledger syncs could form an unbounded backlog -> single-flight guard keyed by pubkey, one all-agents call on startup instead of one spawn per agent -> in_flight_sync_guard_coalesces_concurrent_calls
18. alert suppression was committed before notification delivery -> record_alerts only fires from a new record_delivered_alerts command after the TS caller confirms delivery, wrapped in one transaction -> record_alerts_transactional_failure_rolls_back_all, deliversDesktopNotificationForAlertsOnSyncAndIngest, failedDesktopNotificationDoesNotAckDeliveredAlert
19. raw process log text could leak credentials via notifications -> sanitize_last_error redacts Bearer/api_key patterns before class/payload reach alert evaluation -> sanitize_last_error_redacts_bearer_and_api_key
20. malformed SQLite payloads were treated as healthy -> an agent_paused payload parse failure now reports a distinguishable degraded marker instead of None -> corrupted_payload_marks_degraded
21. alert-state retention errors were explicitly discarded (let _ =) -> prune() wraps both the alert_state and health_events retention deletes in one transaction, propagating errors -> prune_rolls_back_health_events_when_alert_state_delete_fails
22. read paths created state dirs/park files for arbitrary agent IDs -> shared agent_may_have_local_state gate (hex64 + managed-agent roster membership) before every managed_agent_state_dir call at all 3 new call sites -> agent_may_have_local_state_requires_hex64_and_membership
23. CLI converted ledger I/O failures into empty rows -> cmd_health_to_writer now reports a distinguishable "error" state with last_error_class on a read failure, not a healthy-looking offline row -> health_unreadable_ledger_reports_error_state_not_offline
28. retention test did not bind auto-prune-on-open -> added a test that closes and reopens the DB through open_db with no direct prune() call -> retention_prunes_on_reopen_via_open_db
29. four observer-emission match arms were untested -> added assertions for batch_needs_review, agent_resumed, breaker_closed, and relay_reconnected -> record_mirrors_health_kinds_to_observer
Also required to reach a green -D warnings gate: fixed 2 pre-existing (T16) clippy violations (queue.rs extend_with_drain, lib.rs needless_option_as_deref) and 1 violation this ticket introduced (agents.rs write_literal), plus a TS typecheck error (missing AgentHealthAlert import) from the prior fix round's item-18b wiring.
Full verification log: /Users/zero-suminc./.claude/goal-state/buzz-wave5-closeout/proof/T17-fix-verify.txt
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz
Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
1. Community boundary crossing (queue + DB never scoped) -> scoped_db_path per (relay,owner) + frontend queue clear/generation fence -> scoped_db_path_separates_by_subdir_relay_and_owner, a community reset mid-delivery stops the stale run... 2. Unbounded/unauthenticated remote health-frame ingest -> payload char cap + local-roster membership gate before insert -> oversized_payload_frame_is_rejected 3. Forged alert acknowledgements -> cap + agent/rule membership filter before recording -> filter_valid_alert_acks_drops_forged_agent_and_forged_rule, record_delivered_alerts_rejects_over_count_acks 4. Byte cap didn't bound the live DB -> WAL checkpoint + propagated VACUUM errors + re-prune after writes -> prune_enforces_row_and_byte_budget_oldest_first (existing, now exercises the refactor) 5. Subsecond events collapsed on dedupe -> Millis-precision timestamps in event_key -> distinct_subsecond_events_do_not_collapse 6. Parked/needs_review were windowed event counts, not keyed batch state -> HashSet-keyed reconciliation (desktop + CLI reducer) -> parked_counts_current_outstanding_batches_not_windowed_events, parked_and_needs_review_are_cleared_by_replay_or_discard 7. Resolved sync responses hid per-agent errors -> exposed errors field in TS DTO + syncError/syncErrorAgents -> fetchAgentHealthSummaryDegradesOnResolvedResponseWithErrors 8. Frame queue dropped on overload/failure -> overflow/terminal counters + fallback resync on exhausted retries -> overflow beyond the per-agent cap is tracked..., persistent ingest failure falls back to a full resync... 9. Notification delivery could hang sync indefinitely -> withTimeout wrapper around delivery + ack -> withTimeoutFallsBackWhenTheInnerPromiseNeverSettles 10. Single-flight coalescing dropped a late request -> dirty-flag rerun via claim_sync_slot/finish_sync_or_rerun -> late_request_during_active_sync_causes_a_rerun_not_a_drop 11. Corrupt breaker payloads silently defaulted to healthy -> fail-safe sentinel scope + no-op on corrupt close -> corrupted_breaker_payload_fails_safe_not_silently_healthy 12. Arbitrary secrets with no known shape leaked into logs -> known_secrets (env_vars + nsec) threaded into sanitize_last_error -> sanitize_last_error_redacts_known_secrets_with_no_recognizable_shape 13. Parked-batch reader had no byte/line/count bounds -> added the same caps the harness's own reader enforces -> read_parked_batches_rejects_line_over_max_line_bytes, _rejects_more_than_max_parked_total 14. Ledger consumers accepted future timestamps/mixed identities -> read_ledger_file_for_agent, wired into desktop sync_ledger + CLI cmd_health -> sync_ledger_ignores_mismatched_agent_and_future_records, health_ignores_records_with_mismatched_embedded_agent 15. Unicode agent id could panic the CLI text-table renderer -> char_boundary_prefix -> health_text_table_truncates_unicode_agent_without_panicking 16. State-dir containment followed symlinks -> reject_symlink gate before create_dir_all/set_permissions -> reject_symlink_refuses_a_symlinked_path 17. Event-kind filter had no count/length cap -> validate_kinds -> oversized_or_over_count_kinds_filter_is_rejected Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Resolves three conflicts left after T16's head moved (fix commits c691edb, 19c4cbe; zs/main merge 09a62b6; d9162cf): - crates/buzz-acp/src/reliability/ledger.rs: kept T16's dangling-line sanitizer (sanitize_dangling_final_line) ahead of T17's public read_ledger_file (renamed from T16's private read_records, same read body), plus T17's read_ledger_file_for_agent. All six tests from both sides kept (three T17 ledger tests, three T16 ledger tests). - crates/buzz-acp/src/reliability/runtime.rs: T16's DiscardOutcome, FinishReplayReport and improved finish_replay/discard were the only side that touched that region and carried through the auto-merge untouched. Hoisted T17's IntoObserverHandle trait (and its two impls) to module level ahead of the test module, then merged T16's and T17's separate `mod tests` into one, keeping every test and de-duplicating the `use` imports. - desktop/src-tauri/src/managed_agents/storage_tests.rs: additive conflict, not covered by the earlier integration's reuse notes. Kept both independent test blocks — T16's reject_symlink_* tests and T17's validate_state_dir_pubkey_* tests — back to back; the matching `use super::{...}` import addition for validate_state_dir_pubkey had already auto-merged cleanly. crates/buzz-acp/src/lib.rs auto-merged cleanly (no conflict markers): both branches' independent startup hooks (T16's reconcile_on_start call sites, T17's ObserverHandle::in_process/with_observer wiring) are present. Gates run post-merge (see docs/plans or goal-state proof log for full output): cargo fmt --check (root + desktop/src-tauri), cargo clippy -D warnings (buzz-acp + buzz-cli, and desktop/src-tauri), cargo test -p buzz-acp reliability (63 passed), cargo test -p buzz-cli health (4 passed), desktop/src-tauri cargo test agent_health (43 passed) plus targeted reject_symlink/validate_state_dir_pubkey checks on the merged storage_tests.rs (2 + 6 passed), the three agentHealth*.test.mjs node suites (14 passed), and pnpm typecheck. All green; only the merge's own conflicts were touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…t (T17) The pre-push rust-tests lane runs the whole buzz-cli suite. The subcommand_names_are_stable and subcommand_counts_are_stable snapshots still listed five agents subcommands; T17 added buzz agents health, so the snapshot now lists six. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
…e-size limit; use truncatePubkey in the Health tab (T17) The pre-push file-size ratchet caps new files at 1500 lines: agent_health.rs was 1596 and agent_health/tests.rs 1923. The alert-acknowledgement path moves to agent_health/acks.rs and the parked-batch views to agent_health/parked.rs, both re-exported so the command registrations in lib.rs do not change. The second half of the tests moves to agent_health/tests_ingest.rs. The pubkey-truncation lint flagged a hand-rolled slice in AgentHealthTab.tsx; it now uses truncatePubkey from shared/lib/pubkey. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>
Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com> # Conflicts: # crates/buzz-acp/src/lib.rs # crates/buzz-acp/src/reliability.rs # crates/buzz-acp/src/reliability/ledger.rs # crates/buzz-acp/src/reliability/runtime.rs # desktop/src-tauri/src/managed_agents/storage.rs # desktop/src-tauri/src/managed_agents/storage_tests.rs
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Sep 8, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Sep 8, 2026
Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com> # Conflicts: # desktop/src-tauri/src/lib.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
T17 agent health, built per
docs/plans/2026-09-06-agent-health-design.mdand stacked on T16 (feat/harness-reliability, PR #35). Enqueue #35 first; this PR retargets tozs/mainafter #35 lands.crates/buzz-acp/src/reliability/health.rs) and exposes a read-only, bounded ledger reader.agent-health.dbper relay and owner: schema, insert-or-ignore on a canonical event key, 30-day retention plus row and byte budgets, ledger sync, summary and events queries, parked-batch views, alert state.buzz agents healthreads the ledger files from disk (no relay).What changed
crates/buzz-acp)reliability/health.rs(new),reliability/ledger.rs,reliability/runtime.rs,lib.rscrates/buzz-cli)commands/agents.rs(health subcommand),lib.rsdesktop/src-tauri)agent_health.rs(+agent_health/acks.rs,agent_health/parked.rs, tests inagent_health/tests.rsandagent_health/tests_ingest.rs),agent_health_alerts.rs,lib.rs,managed_agents/{backend,retention,storage}.rsfeatures/agents/(health hooks, reducers, alert queue, observer relay store,ui/AgentHealthTab.tsx,ui/AgentHealthDrawer.tsx),e2emock bridge,tests/e2e/agent-health.spec.tsGates run
Targeted gates only. No full suite ran locally on purpose; the merge queue runs it.
cargo fmt --all -- --check(root, desktop/src-tauri)cargo clippy -p buzz-acp -p buzz-cli --all-targets -- -D warningscargo clippy --all-targets -- -D warnings(desktop/src-tauri)cargo test -p buzz-acp reliabilitycargo test -p buzz-cli healthcargo test agent_health(desktop/src-tauri)node --test agentHealthFrames/Summary/Alerts.test.mjspnpm typecheckpnpm exec playwright test --project=smoke agent-health.spec.tsnode scripts/check-file-sizes.mjs,pnpm check:pubkey-truncationPre-push hook (lefthook) lanes all green on the final push. Two earlier hook attempts failed on load-induced timeouts (
claude_named_adapter_wire_lifecycle_records_prompt_and_cost,a_browser_that_never_reports_an_endpoint_is_bounded_and_its_tree_killed); both pass in isolation. Evidence: goal-stateproof/T17-merge-verify.txt,T17-fix2-verify.txt,T17-tester.txt.Tested base OID (merge-base with
feat/harness-reliability):d9162cfcc7175b754ea161e392a93b1a3c44b0e9.Audits
T17-sol-verified.md); closed ine0d9662c9.d19a17dfc(per-community health db scope, roster gate and payload cap on ingest, bounded alert acks, byte budget enforced on write, subsecond-safe event keys, keyed batch state, per-agent sync errors surfaced, frame-queue overflow counters, bounded notification delivery, rerun-not-drop single flight, fail-safe malformed payloads, secret-aware error sanitizer, bounded park reader, ledger agent/timestamp invariants, char-boundary CLI renderer, symlink-rejecting state dirs, capped kind filter).T17-tester.txt).e0d9662c9(T17-critic.md).Follow-ups (WARN, not blocking)
ledger.rs, predates T17).🤖 Generated with Claude Code
https://claude.ai/code/session_01P8VoJ9givMm44HNc4gx5Gz
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.