Uh oh!
There was an error while loading. Please reload this page.
fix(llm): reject oversized Bedrock event-stream frames - #44649
fix(llm): reject oversized Bedrock event-stream frames#44649dajiaohuang wants to merge 1 commit into
Conversation
AWS documents a 16 MiB maximum event-stream message. The frame prelude declares a 32-bit length, so a malformed response can declare up to 4 GiB. Nothing bounded the wait before this change, causing appendChunk to keep allocating and copying the accumulated buffer. Add MAX_EVENT_STREAM_MESSAGE_LENGTH and reject frames that declare a larger length with a typed error instead of buffering toward a frame that can never complete. Fixesanomalyco#44630
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
The following comment was made by an LLM, it may be inaccurate: Potential duplicate found:
This appears to be directly addressing the same issue as PR #44649. Both PRs are focused on rejecting oversized Bedrock event-stream frames that exceed the 16 MiB maximum documented by AWS. You should check if PR #44631 is already merged or if one of these should be closed in favor of the other. |
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
Summary
AWS documents a 16 MiB maximum event-stream message. The frame prelude declares a 32-bit length, so a malformed response can declare up to 4 GiB. Nothing bounded the wait before this change, causing
appendChunkto keep allocating and copying the accumulated buffer.Problem
In
packages/llm/src/protocols/bedrock-event-stream.ts,consumeFramesreadstotalLengthfrom the 4-byte frame prelude and loops until that many bytes have arrived. A malformed response can declare0xFFFFFFFF(4 GiB). Because the buffer is only compacted once a frame completes,appendChunkre-copies the whole accumulated window on every network chunk. Cost is quadratic in bytes received.AWS documents a 16 MiB maximum event-stream message;
@smithy/eventstream-codecdoes not enforce it, and neither did we.Fix
Add
MAX_EVENT_STREAM_MESSAGE_LENGTH = 16 * 1024 * 1024and reject frames that declare a larger length with a typedInvalidProviderOutputerror instead of buffering toward a frame that can never complete.Testing
After the fix, a frame declaring > 16 MiB immediately emits an error and stops buffering.
Fixes#44630