Skip to content

buzz-acp: per-channel cancelled-batch side-map is uncapped — sustained mid-turn interrupts grow harness memory and the LLM prompt without bound #3042

Description

@shani-singh1

Summary

The ACP harness keeps a per-channel cancelled_batches side-map that is never size-capped, unlike the main event queue beside it. Under a sustained mid-turn interrupt/steer flood — every turn cancelled before it completes — the map, and the LLM prompt rendered from it, grow without bound: the agent process eventually OOMs, and every subsequent turn's prompt is amplified by the full content of every previously-cancelled event.

Found on main @ 1a56b7cc by source read + a unit test that drives the cancel cycle.

The asymmetry

The main queue is capped at MAX_PENDING_PER_CHANNEL (500) in every growth path — push (queue.rs:242), requeue (:488), requeue_preserve_timestamps (:521), and others (:722, :775), each with a while len > cap { pop; warn }.

requeue_as_cancelled (crates/buzz-acp/src/queue.rs:542) has no such cap:

pub fn requeue_as_cancelled(&mut self, batch: FlushBatch, reason: CancelReason) {
    let entry = self.cancelled_batches.entry(batch.channel_id).or_default();
    entry.extend(batch.cancelled_events);  // prior accumulated cancelled events
    entry.extend(batch.events);            // + the just-cancelled turn's events
    self.cancel_reasons.insert(batch.channel_id, reason);
}

cancelled_batches[ch] is cleared only when a turn completes (it's removed in flush_next and not re-stored) or when the channel drains (drain_channel, :633). So while turns keep getting cancelled, it only grows.

format_prompt (queue.rs:1505) then renders every accumulated event into the prompt on each new turn:

for (i, be) in batch.cancelled_events.iter().enumerate() {   // unbounded
    s.push_str(&format!("\n\n--- Event {} ({}) ---\n{}", i + 1, be.prompt_tag,
        format_event_block(batch.channel_id, ..., be, ...)));
}

How it accumulates

Each cancel cycle: flush_next moves cancelled_batches[ch] into the new batch's cancelled_events; if that turn is also cancelled, requeue_as_cancelled stores cancelled_events (the prior set) plus the turn's events back. Net: +1 turn's events per cancel, forever.

Reproduction

Config: --multiple-event-handling=interrupt (or steer), which forces --dedup=queue, and a respond_to that admits the sender — anyone on a public channel, or an allowlist that includes the sender.

  1. Send a mentioning kind:9 in channel C → turn T1 starts with batch [E1].
  2. While T1 runs, send E2. It passes the author gate, is queued, and mode_gate_signal fires Interrupt/Steer → T1 is cancelled → cancelled_batches[C] = [E1].
  3. flush_next starts T2 = {events:[E2], cancelled_events:[E1]}.
  4. Send E3 → T2 cancelled → cancelled_batches[C] = [E1, E2].
  5. Repeat. After K cancels, cancelled_batches[C] holds K events, and each new turn's prompt embeds all K.

K is bounded only by how long the flood is sustained.

I reproduced the growth in a unit test that runs the cancel cycle MAX_PENDING_PER_CHANNEL + 50 times: without a cap, cancelled_batches[ch] reaches 550; the test asserting <= 500 fails.

Impact

  • Memory: unbounded harness growth → OOM of the agent process.
  • Cost/DoS: every subsequent turn's prompt carries the full content + tags of every cancelled event → token-cost amplification and effective denial of useful service on the channel.
  • Reachability: with respond_to = anyone (public channel) any participant triggers it; with allowlist, one misbehaving/compromised allowed key. Under the default owner-only it is only self-inflicted.

Suggested fix

Cap cancelled_batches[channel] exactly as the main queue is capped — trim oldest beyond MAX_PENDING_PER_CHANNEL after the two extends, keeping the most-recent "what you were working on" events. PR attached.

(Found while auditing the agent harness. The authorization surfaces in the same area — the inbound author gate, NIP-OA sibling verification, !shutdown/!cancel/!rotate owner checks, and the owner→agent control-frame gating — all held up under tracing; this queue cap was the one reachable resource issue.)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions