fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559

Merged
moonming merged 2 commits into
mainfrom
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged

fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom
fix/chat-streaming-toolcall-scan

Conversation

@moonming

@moonmingmoonming commented Jun 8, 2026

Copy link
Copy Markdown
Member

What

On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.

How

Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.

Test plan

A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.

Closes#558. Refs #719, #546, #448.

@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai

coderabbitaiBot commented Jun 8, 2026•
edited
Loading

Copy link
Copy Markdown

Warning

Review limit reached

@moonming, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 5a364530-db01-4c84-a4ab-4209beff8034

📥 Commits

Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.

📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs

Note

🎁 Summarized by CodeRabbit Free

Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.

Comment @coderabbitai help to get the list of available commands and usage tips.

@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming

moonming commented Jun 8, 2026

Copy link
Copy Markdown
MemberAuthor

Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:

  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.

The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.

@moonming
moonming merged commit 634ab4c into mainJun 8, 2026
8 checks passed
@moonming
moonming deleted the fix/chat-streaming-toolcall-scan branch June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)

1 participant

@moonming
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
fix(guardrails): scan tool-call args on chat.rs streaming output (#558) - #559
Merged
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Jun 8, 2026
Merged
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)#559
moonming merged 2 commits into
mainfrom 
fix/chat-streaming-toolcall-scan
Conversation
@moonming
@moonmingmoonming
commented
Jun 8, 2026

Copy link


Copy Markdown

Member
What
On /v1/chat/completionsstreaming, the output guardrail buffered only delta.content for the end-of-stream check_output; delta.tool_calls was never accumulated. So a blocked literal in a streamed tool-call's function.arguments bypassed the output guardrail. This is asymmetric: chat non-streaming scans tool-call output (ChatResponse::guardrail_output_text, the #448 fix), and /v1/messagesstreaming scans them too — only chat streaming leaked. Surfaced by the independent pre-fix audit of the #719 follow-up batch.
How
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the stream (bounded to DEFAULT_STREAM_OUTPUT_BUFFER_BYTES; concatenated without a separator so a literal split across deltas reassembles), and merge them into the scanned text at both end-of-stream check sites (the hold-back BufferFull/Window path and the EndOfStreamCheck path). Under the default BufferFull policy the tool-call chunks are already held in pending (line ~2399), so a block holds them back — they never reach the wire.
Test plan
A streamed tool_call delta carrying a blocked literal in arguments blocks the stream (event: error with error.type: content_filter) and the args never appear on the wire (held back). Mirrors the existing streaming_output_guardrail_blocks_with_sse_error_event_and_no_done test. fmt + clippy clean; 403 aisix-proxy lib tests pass.
Closes#558. Refs #719, #546, #448.
@moonming
fix(guardrails): scan tool-call args on chat.rs streaming output (#558)
a33bef2
chat.rs streaming buffered only delta.content for the end-of-stream
check_output, so a blocked literal in a streamed tool-call's
function.arguments bypassed the output guardrail — asymmetric with chat
non-streaming (guardrail_output_text, #448) and /v1/messages streaming,
which both scan tool calls.
Accumulate tool-call name + arguments from chunk.delta.tool_calls across the
stream (bounded to the hold-back cap; concatenated without a separator so a
literal split across deltas reassembles) and merge them into the scanned
text at both end-of-stream check sites (BufferFull/Window hold-back and
EndOfStreamCheck). Under the default BufferFull policy the tool-call chunks
are already held in pending, so a block holds them back (never on the wire).
Test: a streamed tool_call delta with a blocked literal in arguments blocks
the stream (event: error / content_filter) and the args never reach the wire.
fmt + clippy clean; 403 lib tests pass.
Closes#558.
@coderabbitai
coderabbitaiBot
commented
Jun 8, 2026•

edited

Loading

Copy link


Copy Markdown

Warning
Review limit reached
@moonming, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 15 minutes and 53 seconds. Learn how PR review limits work.
Your organization has run out of usage credits. Purchase more in the billing tab.
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Free
Run ID: 5a364530-db01-4c84-a4ab-4209beff8034
📥 Commits
Reviewing files that changed from the base of the PR and between bc369e1 and 02bd30d.
📒 Files selected for processing (2)
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/lib.rs
Note
🎁 Summarized by CodeRabbit Free
Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.
Comment @coderabbitai help to get the list of available commands and usage tips.
@moonming
test+fix(guardrails): add the streaming tool-call test; close Window-…
02bd30d
…mode residual (#558)
Audit of #559 found two issues:
- HIGH-1: the fix shipped with NO test (the prior commit ran `git add chat.rs`
only, so the lib.rs test was never committed — yet the PR/commit claimed one).
Add streaming_output_guardrail_blocks_tool_call_arguments: a tool-call-only
stream with a blocked literal in function.arguments -> event: error /
content_filter, and the args never reach the wire (RED without the fix,
GREEN with it).
- MEDIUM-1: Window mode released a content window (and its held tool-call
chunks) BEFORE the tool-call text was scanned (scanned only at end-of-stream
-> too late). Fold tool_calls_buf into the per-window scan so a blocked
tool-call arg is caught before its events are released.
fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
commented
Jun 8, 2026

Copy link


Copy Markdown

MemberAuthor
Audit of this PR returned DON'T-MERGE on two items, both now addressed in the latest commit:
  • HIGH-1 (no test; the PR/commit text falsely claimed one — caused by a git add chat.rs-only slip): fixed — added streaming_output_guardrail_blocks_tool_call_arguments (tool-call-only stream, blocked literal in arguments → event: error/content_filter, args never on the wire; RED-without-fix / GREEN-with-fix).
  • MEDIUM-1 (Window-mode residual: tool-call chunks released per content-window before the tool-call text was scanned): fixed — the per-window scan now folds in tool_calls_buf, so a blocked tool-call arg is caught before its held events are released.
  • LOW-1 (tool-call buffer overflow truncates vs content's fail-closed cap) — extreme edge (>256 KiB args); noted, not addressed.
The audit verified the default BufferFull path was already correct (real hold-back, not luck). fmt + clippy clean; 403 lib tests pass.
@moonming
moonming
merged commit 634ab4c
into
mainJun 8, 2026

8 checks passed

@moonming
moonming
deleted the

fix/chat-streaming-toolcall-scan

branch
June 8, 2026 08:00
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

chat.rs streaming output guardrail doesn't scan tool-call args (#448 leak on streaming)


1 participant

@moonming