improvement(providers): align attachment dispatch to vendor SDK types

[waleedlatif1]

Summary

• Restore xAI Grok vision via OpenAI-compatible chat-completions (image_url parts)
• Fix Azure OpenAI chat-completions vision; documents still require Responses API
• Replace loose Record<string, unknown> wire shapes with satisfies clauses against each vendor SDK union (OpenAI Responses/Chat, Anthropic ContentBlockParam, Gemini Part, Bedrock ContentBlock)
• AnthropicImageMediaType now derives from Base64ImageSource['media_type'] so it tracks SDK updates
• Collapse provider validation cascade into an exhaustive switch with never enforcement
• Drop redundant formatMessagesForProvider call from xai/index.ts (providers/index.ts already dispatches centrally)

Type of Change

• Improvement

Testing

Tested manually. Vitest suites for providers/attachments, providers/openai/utils, providers/google/utils, executor/handlers/agent/agent-handler, executor/handlers/agent/memory, and blocks/blocks all pass (182/182). Biome clean.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		May 15, 2026 10:21pm

[cursor]

PR Summary

Medium Risk
Moderate risk because it changes how attachments are validated and serialized across multiple provider adapters (OpenAI/Anthropic/Gemini/Bedrock/OpenRouter/xAI/Azure), which could break multimodal requests if any SDK shape expectations are off.

Overview
Aligns attachment handling to vendor SDK-native content union types across providers, replacing loose Record<string, unknown> parts with satisfies-checked OpenAI Responses/Chat, Anthropic content blocks, Gemini Part, and Bedrock ContentBlock shapes (including adding OpenAI detail: 'auto' for images).

Refactors provider MIME-type support validation into an exhaustive switch and updates provider capability messaging, notably restoring xAI Grok vision as image-only and tightening unsupported-provider behavior.

Fixes Azure OpenAI chat-completions vision by converting user image attachments into image_url content parts while explicitly rejecting non-image files unless using the Responses API endpoint.

^{Reviewed by Cursor Bugbot for commit af6f058. Configure here.}

[greptile-apps]

Greptile Summary

This PR aligns provider attachment dispatch to vendor SDK types by replacing loose Record<string, unknown> wire shapes with satisfies clauses validated against each vendor's SDK union types, restores xAI Grok vision via chat-completions image_url parts, and upgrades Azure OpenAI's chat-completions path to properly handle image attachments rather than unconditionally throwing.

• Type safety: satisfies clauses are applied against OpenAI.Responses.ResponseInputContent, OpenAI.Chat.Completions.ChatCompletionContentPart, Anthropic.Messages.ContentBlockParam, @google/genai Part, and @aws-sdk/client-bedrock-runtime ContentBlock — catching shape mismatches at compile time and adding detail: 'auto' to Responses API image parts.
• Provider validation refactor: The nested ternary chain in validateProviderSupport is replaced by an exhaustive switch in isMimeTypeSupportedByProvider with a never-enforced default, and formatMessagesForProvider gains an early-return for SDK-native providers (openai/anthropic/google/bedrock) that perform their own message construction.
• Azure OpenAI vision fix: The chat-completions path now converts image files to image_url content parts per message instead of rejecting all file attachments; non-image files route to a clear error directing users to the Responses API endpoint.

Confidence Score: 5/5

Safe to merge — changes are well-scoped type-safety improvements with no regressions in attachment dispatch logic.

The refactor is mechanical and verifiable: loose Record shapes become SDK-typed satisfies clauses that would have failed at compile time if shapes were wrong, the exhaustive switch is TypeScript-enforced, and the Azure OpenAI chat-completions vision path is a straightforward per-message translation that mirrors the existing pattern in other providers. All previously flagged issues are confirmed fixed in this HEAD. Tests pass at 182/182.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/providers/attachments.ts	Core attachment logic refactored: loose Record shapes replaced with SDK-typed satisfies clauses, IMAGE_ONLY_PROVIDERS removed in favor of exhaustive switch with never enforcement, formatMessagesForProvider gains early-return for SDK-native providers, xai added to image-only case. All changes are correct and consistent.
apps/sim/providers/azure-openai/index.ts	Chat-completions path now builds image_url content parts per user message with files instead of unconditionally throwing; non-image files get a clear actionable error. Non-user messages with files are forwarded as-is (expected — assistant messages never carry files in practice).
apps/sim/providers/attachments.test.ts	Test updated to expect detail: 'auto' on input_image parts, matching the new satisfies OpenAI.Responses.ResponseInputImage shape.
apps/sim/providers/openai/utils.test.ts	Test updated to expect detail: 'auto' on input_image from buildResponsesInputFromMessages, consistent with the attachments change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[ProviderRequest with files] --> B{Provider type?}
B -->|openai / anthropic\ngoogle / bedrock| C[SDK-native path\nformatMessagesForProvider\nearly-returns unchanged]
C --> D[Provider builds own\ncontent blocks\nbuildOpenAIMessageContent\nbuildAnthropicMessageContent etc.]
B -->|xai / mistral / groq\nfireworks / ollama / vllm| E[formatMessagesForProvider\nbuildOpenAICompatibleChatContent]
E --> F[image_url chat\ncompletion parts\nimages only]
B -->|openrouter| G[formatMessagesForProvider\nbuildOpenRouterMessageContent]
G --> H[image_url + file\nparts mixed]
B -->|deepseek / cerebras| I[throw: unsupported]
B -->|azure-openai\nchat-completions path| J{message.role === user\n&& has files?}
J -->|No| K[push message as-is]
J -->|Yes| L[prepareProviderAttachments\nazure-openai]
L --> M{non-image\nattachment?}
M -->|Yes| N[throw: use\nResponses API]
M -->|No| O[image_url parts\nfor chat-completions]
B -->|azure-openai\nResponses API path| P[executeResponsesProviderRequest\nbuildOpenAIMessageContent]

Loading

_{Reviews (2): Last reviewed commit: "fix(providers): restore getProviderAttac..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit af6f058. Configure here.}