feat(acp): show where reply time goes after an agent @mention

[BradGroux]

Note: This is a replacement for PR #2408, which was accidentally closed when branches were force-pushed after a commit identity rewrite. The changes are identical, rebased onto the latest main.

Refs #2386.

Why this PR exists

Four simple agent replies in #2386 took 13–31 seconds, but the available event timestamps only showed the beginning and end of each turn. That left no way to tell whether the delay happened in queueing, runtime startup, model work, or reply publication.

This PR adds the first in-process timing slice after Buzz receives the mention.

What changed

• Record process-local monotonic boundaries for relay receipt, queue admission, ACP session resolution, prompt dispatch, first semantic model or tool output, first reply fan-out, and turn completion.
• Correlate those boundaries into content-free mention_reply_latency samples.
• Report rolling warm/cold p50, p95, and max summaries for each stage.
• Let observer-enabled mention subscriptions receive self-authored kind-9 replies for telemetry while preserving the normal self-dispatch guard.
• Prefer signed parent/root correlation over whichever turn is currently active, so delayed relay fanout cannot be assigned to a newer turn.
• Treat unmatched signed thread tags as authoritative negative evidence instead of falling back to the active turn.
• Use channel fallback only for replies with no signed thread tags, and fail closed when more than one turn is possible.
• Document a deterministic check and an opt-in warm/cold live benchmark.

What this measures

This slice measures harness_relay_receipt through harness_relay_fanout. It does not claim sender-publish-to-recipient-render timing.

Durations come from one process-local monotonic clock. RFC3339 and Nostr timestamps are correlation metadata only.

The new semantic timing events contain identifiers, path classification, durations, and sample counts. They do not include message content, prompts, model output, credentials, or tool arguments. Existing raw acp_read and acp_write observer events are unchanged and remain outside this new redaction guarantee.

Verification

• cargo fmt --all -- --check
• cargo clippy -p buzz-acp --all-targets --all-features -- -D warnings
• cargo test -p buzz-acp --no-fail-fast with ambient lazy-pool overrides removed: 673 library tests and 9 pool-lifecycle tests passed.
• just ci passed formatting, workspace strict Clippy, Desktop check, Tauri strict Clippy, web check, mobile format/analyze, Rust suites, 3,885 Desktop tests and build, 2,047 Tauri library tests plus 3 diagnostics, and the web build. The final mobile suite reached 1,021 passed and 1 skipped, then reproduced the unchanged current-main failure ChannelDetailPage keeps follow mode off while a tall newest message stays visible at mobile/test/features/channels/channel_detail_page_test.dart:1053. This branch has no mobile diff.

What remains

#2386 should stay open until the outer sender-publication and recipient-render or relay-acceptance boundaries are measured, hosted warm/cold baselines and explicit budgets exist, and a scheduled or blocking regression job is in place.

[BradGroux]

This is a replacement for #2408, which was accidentally closed when branches were force-pushed after a commit identity rewrite. Prior review comments and feedback are preserved on the old PR:

#2408

[BradGroux]

Rebased onto the latest main (was 36 commits behind). The branch applied cleanly. The mention-to-reply latency tracing feature is not present on current main, so this PR is still needed.

No review comments on this PR. CI will verify the build and test gate.

[github-actions]

🔐 Codex Security Review

Status: review required for the current range.

The current range is 213092074c269bee90c87163672f9c2b8d714cfe...a82b2dcf878ea10fe769bb10d2007a6b7fb62865.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review a82b2dcf878ea10fe769bb10d2007a6b7fb62865 to authorize a new review.
Any previous review applies only to its recorded range.

[BradGroux]

Rebased onto the latest main (was 62 commits behind at f249710, now at 00e61ea). The branch applies cleanly with all tests passing.

Still-needed assessment: No upstream commits in the 62-commit window address the same issue. This PR remains needed.

Review requests: buzz-oss-team review request is active from the previous pass.

[BradGroux]

Rebased onto current block:main (2af9773d6) — 38 new commits since the last rebase pass (Aug 29, 00e61eafa).

Conflict resolved: crates/buzz-acp/src/lib.rs — Upstream PR #6732 (thread-scoped ACP sessions) refactored the normal-listener event path into AuthorizedNormalListenerEvent → match_subscription → NormalListenerIngress::push → QueuedNormalListenerEvent with mark_seen() and steer_or_interrupt() methods, replacing the old inline queue.push() + accepted boolean + manual reaction/steer code. This PR's observer event_received and event_queued emits were originally interleaved with that old inline code. The resolution moves the observer emits to work with the new struct-based flow: event_received is emitted before ingress.push(), and event_queued is emitted after, gated on queued.accepted. The event_received_at timing variable (captured earlier in the relay handler) is preserved.

Still needed: Upstream #6732 adds per-thread session scoping and #6953 wakes agents from workflow messages — neither addresses mention-to-reply latency tracing. The PR remains relevant.

Verification:

• Branch is 0 commits behind block:main, mergeable on GitHub
• All commits authored as Brad Groux (3053586+BradGroux noreply)
• DCO signoff preserved on all commits

[BradGroux]

Rebase and review — 2026-09-03

Base commit: 752cbfc03 (latest main)
Mergeable: Yes
Conflict resolution: Main refactored the ACP event handling loop (introduced AuthorizedNormalListenerEvent, SessionScope, QueuedNormalListenerEvent with mark_seen/steer_or_interrupt methods). Merged the PR's latency tracing (observer event_received/event_queued emissions) into the refactored code path. Captured event_id_hex, event_created_at, and event_channel_id before ingress.push() moves the event. cargo check -p buzz-acp passes.

Accuracy review: No mention-to-reply latency tracing exists on main.
Continued relevance: Still needed.

[BradGroux]

Rebase and review update — Sep 3, 2026

Rebased onto latest main (88687876f). No conflicts.

Accuracy review: Main does not have a latency collector for mention-to-reply tracing. The buzz-acp crate has observer infrastructure but no latency.rs module, no mention_reply_latency collector, and no event_received/event_queued emission at the inbound admission path. The PR's feature to trace where reply time goes after an agent @mention is still needed.

Mergeable status: Confirmed MERGEABLE on GitHub after force-push.

[BradGroux]

Rebase pass — Sep 5, 2025

Rebased onto current upstream main (f038cbbb0). Main moved 62 commits since the Sep 2 rebase pass (2af9773d6).

Rebase result

Rebased cleanly onto current main. No conflicts.

Still-needed assessment

Checked all 62 new main commits for overlap with this PR's topic. None obsoleted:

• Verify ACP relay events before prompt routing #7010 (verify ACP relay events before prompt routing) adds signature verification to relay event handling — does not address SubscribeMode filtering (fix(acp): stop SubscribeMode::All from subscribing to every event kind #6646) or mention latency tracing (feat(acp): show where reply time goes after an agent @mention #6637)
• fix(buzz-acp): bound busy-owner hold to prevent cross-channel starvation #7337 (bound busy-owner hold to prevent cross-channel starvation) changes ACP dispatch scheduling — does not add latency observers (feat(acp): show where reply time goes after an agent @mention #6637)
• fix(acp): rename system tag to agent-instructions #7332 (rename system tag to agent-instructions) renames prompt boundary tags — does not affect MCP server wiring (feat(acp): accept extra MCP servers via BUZZ_ACP_EXTRA_MCP_COMMANDS #6651, fix(acp): add BUZZ_ACP_AGENT_ARGS_JSON for lossless structured argument transport #6654, fix(desktop): wire MCP server for Claude Code agents #6658) or agent args transport (fix(acp): add BUZZ_ACP_AGENT_ARGS_JSON for lossless structured argument transport #6654)
• perf(desktop): publish mention sends before waking agents #7154 (publish mention sends before waking agents) is a desktop send-path optimization — does not address ACP latency observability (feat(acp): show where reply time goes after an agent @mention #6637)
• feat(agents): harness-agnostic effort write path and spawn bridge #4625 (harness-agnostic effort write path) extracts runtime catalog to catalog.rs and adds effort normalization — does not fix Claude MCP wiring (fix(desktop): wire MCP server for Claude Code agents #6658, still has mcp_command: None upstream), Goose provider URL translation (fix(desktop): let Goose agents use OpenAI-compatible provider URLs #6641), or capability readiness manifests (feat(agent): show readiness and capabilities before delegation #6643)
• feat(cli): add buzz gifs command group and NIP-30 emoji tags on messages #7259 (buzz gifs command group and NIP-30 emoji tags) adds emoji tag support to build_message and new CLI commands — does not address NUL byte validation (fix(cli): reject NUL bytes in message content instead of silently truncating #6653), thread root filter (fix(cli): apply --channel to the root filter in messages thread #6655), or include_aux (fix(cli): plumb include_aux so messages thread/get can see reactions #6656)
• refactor(relay): extract NIP-29 membership authorization #7285 (extract NIP-29 membership authorization) refactors relay authz — does not affect multi-channel HTTP query (fix(relay): stop multi-channel HTTP query from narrowing to one UUID #6648)
• fix(desktop): authorize remote mentions at publication #7124, fix(desktop): bind duplicate mention selections to exact recipients #7133, fix(desktop): restore mention chip identity icons #7338 (mention chip identity, duplicate selections, authorize remote mentions) are desktop mention UI changes — do not affect SDK self-targeted p tags (fix(sdk): preserve self-targeted p tags in add-member, remove-member, and DM builders #6657)

Verification

• PR confirmed MERGEABLE on GitHub
• All commits authored as Brad Groux (3053586+BradGroux noreply)
• DCO signoff preserved on all commits
• Review requests to buzz-oss-team still active from the Aug 24 recovery pass

[BradGroux]

Restoration note — Sep 16, 2026: The original text of this comment was accidentally overwritten when the Sep 16 rebase pass posted its follow-up with an edit flag instead of as a new comment. What follows is a faithful reconstruction from the session record; the byte-exact original remains in this comment's GitHub edit history (hover the edited marker on this comment).

Rebase pass — Sep 9, 2026

Rebased onto current upstream main (c045321a7). Main moved 8 commits since the Sep 5 rebase pass (f038cbbb0).

Rebase result

Conflict resolved in crates/buzz-acp/src/relay.rs — upstream #7340 added HoldDeadline wakeup logic and refactored dispatch_pending in lib.rs, which changed the test module in relay.rs. The PR added two test functions and a seed_test_subscription helper in the same region. Resolution kept both the PR tests and upstream's updated pub(super) visibility.

Still-needed assessment

Checked all 8 new main commits for overlap with this PR's topic. None obsoleted. #7340 (wake held ACP threads) changes lib.rs/pool.rs/queue.rs and #7325 (pace targeted overflow recovery) changes relay.rs recovery paths — neither adds latency observers for agent mentions.

Verification

• PR confirmed MERGEABLE on GitHub
• All commits authored as Brad Groux (3053586+BradGroux noreply)
• DCO signoff preserved on all commits
• Review requests to buzz-oss-team still active from the Aug 24 recovery pass

[BradGroux]

Rebase pass — Sep 16, 2026

Rebased onto current upstream main (213092074). Main moved 37 commits since the Sep 9 rebase pass (c045321a7).

Rebase result

Rebased cleanly onto current main. No conflicts.

Still-needed assessment

None of the 37 new main commits add latency observability for agent mentions. #7620 (deduplicate thread context by ACP session delivery) changes session delivery accounting but does not add per-stage latency spans.

Verification

• PR confirmed MERGEABLE on GitHub
• All commits authored as Brad Groux (3053586+BradGroux noreply)
• DCO signoff preserved on all commits
• buzz-oss-team review request still active

What was not done

No local test suites were run. The rebase only repositions commits; conflict resolutions carry the same logical changes forward (CI will verify builds).