Skip to content

fix verbatim-repeat false positive on auth-retry; add subagent thrash detail - #295

Open
EdbertChan wants to merge 1 commit into
mainfrom
fix/token-audit-verbatim-repeat-and-subagent-detail
Open

fix verbatim-repeat false positive on auth-retry; add subagent thrash detail#295
EdbertChan wants to merge 1 commit into
mainfrom
fix/token-audit-verbatim-repeat-and-subagent-detail

Conversation

@EdbertChan

@EdbertChanEdbertChan commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Summary

The verbatim-repeat detector in token_audit.py flagged a user re-sending the same prompt after an OAuth 401 failure as frustrated restatement. The user interrupted, ran /login, and re-sent -- a legitimate retry, not frustration. The detector now checks whether a successful assistant response occurred between two identical sends. When no assistant turn happened between them, the re-send is suppressed as a retry.

Separately, the subagent thrash section in --out JSON reported counts and flag names but not which files were redundantly read or what the recurring failure signatures were. The reflect lenses could not act on the finding without that detail.

Review Claim

These two mechanical detection fixes reduce false positives in the frustration pipeline and make subagent thrash findings actionable.

Review Lane

behavior

Safety Invariant

Non-Claude modes (Codex, OMP, Cursor) do not pass assistant_turn_indices, so they retain the existing verbatim-repeat behavior. The user_msgs ordinal changed from enumerate-counter to JSONL line index in Claude mode, but this only affects the "index" field in flagged output (reporting, not logic).

Slice Rationale

Both fixes target the same file (token_audit.py) and the same audit pipeline. Splitting them would create two PRs that touch the same function signatures.

Non-goals

No changes to the frustration lens itself, to cat-mode SKILL.md, or to any other reflect script.

Test Plan
  • cd engine/skills/reflect/scripts && python3 -m pytest tests/test_token_audit.py -v -- 72 passed, 6 subtests passed
  • test_verbatim_repeat_suppressed_when_assistant_responded_between_sends -- covers both no-assistant (suppressed) and with-assistant (flagged) cases
  • test_verbatim_repeat_still_works_for_non_claude_modes -- backward compat
  • test_subagent_thrash_detail_includes_files_and_failure_signatures -- verifies redundant_read_files and recurring_failure_details populated
Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <sha>
  • Post-revert steps: None
  • Data migration? No

Note

Low Risk
Changes are confined to reflect token_audit heuristics and report shape; non-Claude paths are unchanged and behavior is covered by unit tests.

Overview
Fixes false-positive verbatim-repeat frustration signals and enriches subagent thrash JSON for reflect audits.

Claude token_audit now passes JSONL line indices for human utterances and tracks where successful assistant turns occur. frustration_signals accepts optional assistant_turn_indices and uses _has_assistant_between so identical user prompts re-sent within 10 minutes without an assistant response in between (e.g. OAuth 401 → login → retry) no longer count as verbatim-repeat; repeats after an assistant reply still do. Codex/OMP/Cursor keep prior behavior because they do not pass that list.

Audit output adds redundant_read_files and recurring_failure_details on Claude sessions; audit_subagents rolls them into subagents.thrash as agent:path file lists and per-agent recurring failure rows so --out reports are actionable beyond counts.

Tests cover suppression vs real repeats, non-Claude backward compatibility, and subagent thrash detail fields.

Reviewed by Cursor Bugbot for commit 962090f. Bugbot is set up for automated code reviews on this repo. Configure here.

… thrash detail
## Context
- Problem: Two false positives in token_audit.py's detection pipeline.
- Trigger: reflect-on-thrash hook fired on a session where the user's prompt
hit an OAuth 401 (4 retries, no assistant response), the user interrupted,
ran /login, and re-sent the same prompt. The verbatim-repeat detector
flagged this as frustrated restatement instead of a legitimate retry after
auth failure.
## Considerations
- R1 (verbatim-repeat): The detector compared consecutive user messages
without checking whether the agent ever responded between them. A re-send
after an API error (no successful assistant turn) is a retry, not
frustration. Fix: pass assistant JSONL line indices into
frustration_signals(); suppress verbatim-repeat when no assistant turn
occurred between the two sends. Non-Claude modes (Codex, OMP, Cursor) do
not pass this parameter, so they retain the existing behavior.
- B2 (subagent thrash detail): The --out JSON reported counts and flag names
but not which files were redundantly read or what the recurring failure
error messages were. Without specifics, the reflect lenses cannot act on
the finding. Fix: expose redundant_read_files and
recurring_failure_details in audit_claude()'s return dict, aggregate them
per-agent in audit_subagents().
- Alternative for R1: could have checked timestamps of API error responses
between user messages, but that requires parsing error shapes per-harness.
Checking for the presence of a successful assistant turn is simpler and
covers all error types (OAuth, rate limit, network).
## Blast Radius
- Files touched: token_audit.py, test_token_audit.py
- Risks: Changed the user_msgs ordinal from enumerate-counter to JSONL line
index in Claude mode. This only affects the "index" field in flagged
frustration output (used for reporting, not for logic elsewhere). Other
modes are unchanged.
## Verification
- 72 tests pass (69 original + 3 new), 6 subtests pass.
- New tests cover: suppression when no assistant between sends, no
suppression when assistant did respond, backward compat for non-Claude
modes, subagent detail fields populated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cursor

cursorBot commented Sep 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a4af20cc-5047-4b9b-bd3e-f9bdfcfd731a)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@EdbertChan@edbert-bot