Uh oh!
There was an error while loading. Please reload this page.
feat(opencode): add native video and audio file reading support - #18005
feat(opencode): add native video and audio file reading support#18005mugnimaestra wants to merge 2 commits into
Conversation
- Extend read tool to handle video/* and audio/* MIME types as base64 attachments (20MB cap) - Patch @ai-sdk/openai-compatible to send video as video_url type and audio as input_audio type - Extend isMedia() in message-v2 to recognize video and audio for proper compaction handling - Binary file check runs before video/audio MIME detection to prevent .ts file misdetection
mugnimaestra
commented
Mar 18, 2026
👋 Hey @kitlangton@adamdotdevin@rekram1-node — this implements a community-requested feature (#10531, 7 upvotes) for native video and audio reading support. It follows the exact same pattern as existing PDF/image support in the |
mugnimaestra
commented
Mar 18, 2026
💡 Workaround for files over 20MBIf your video exceeds the 20MB limit, you can use the video-frames skill as an alternative — it extracts key frames from video files using ffmpeg for LLM analysis, without needing to attach the full video. Install it with: npx skills add mugnimaestra/video-frames-skillThis lets you analyze large videos by converting them into representative JPEG frames that fit well within context limits. |
fengwk
commented
Mar 31, 2026
This is a very important feature |
ayulockin
commented
Apr 30, 2026
This is a nice feature addition. Is there any ETA? |
MuziCodes
commented
Jun 10, 2026
It's very clear that we all need it. It shouldn't be so hard to implement. |
MoodyMarshmallow
commented
Jun 14, 2026
+1 on this. Would really like to see this merged. |
Issue for this PR
Closes#10531
Type of change
What does this PR do?
Adds native video and audio file reading to the
readtool. When a model-capable provider (e.g. Kimi K2.5 via Chutes) asks to read a video or audio file, it's base64-encoded and returned as a file attachment — same pattern as existing PDF/image support.Three changes:
read.ts — Detect video/* and audio/* MIME types on binary files, return as base64 attachment (20MB cap). Moved the binary file check before MIME detection to prevent
.tsfiles (MIMEvideo/mp2t) from being misdetected.message-v2.ts — Extended
isMedia()to include video/* and audio/* so compaction handles them correctly.@ai-sdk/openai-compatible patch — The bundled v1.0.32 only handles image/* in its chat converter. This patch adds video_url and input_audio types for OpenAI-compatible providers that support video (like Kimi K2.5).
Related: #16338 (same package fails for PDF — a follow-up patch could cover that too).
20MB file size limit
Files over 20MB are rejected with a clear error message. This is the safe lowest-common-denominator across LLM providers:
There is currently no built-in tool to compress oversized media — users can rely on the model invoking
ffmpegvia thebashtool if installed.How did you verify your code works?
bun typecheckpasses with zero errorsScreenshots / recordings
Session transcript: https://gist.github.com/mugnimaestra/fd39a67ca38e0551287c0da93aa20bef
Checklist