feat(tools): added speech to text with openai whisper, elevenlabs, and deepgram

[waleedlatif1]

Summary

• added speech to text with openai whisper, elevenlabs, and deepgram

Type of Change

• Bug fix

Testing

Tested manually

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Preview	Comments	Updated (UTC)
docs	Skipped			Nov 20, 2025 4:41am

[greptile-apps]

Greptile Summary

• Implemented speech-to-text functionality with three AI providers (OpenAI Whisper, Deepgram, ElevenLabs) supporting audio/video transcription with timestamps and speaker diarization
• Added ffmpeg integration for video-to-audio extraction with proper Docker and Next.js configuration
• Extended file upload validation and MIME type handling to support audio formats (mp3, wav, m4a, etc.) and video formats (mp4, mov, avi, mkv)

Confidence Score: 3/5

• This PR has a critical concurrency bug that could cause file conflicts in production under load.
• The temporary file naming in extractor.ts uses only Date.now() which has millisecond precision, creating a race condition where concurrent requests could collide and overwrite each other's files. This will cause transcription failures and potential data corruption under moderate to high traffic.
• Pay close attention to apps/sim/lib/audio/extractor.ts - the temp file collision bug must be fixed before merging.

Important Files Changed

Filename	Overview
apps/sim/lib/audio/extractor.ts	Audio/video extraction using ffmpeg with temp file name collision bug in concurrent requests
apps/sim/app/api/proxy/stt/route.ts	API route handling STT transcription with proper auth, file handling, and provider integration

Sequence Diagram

sequenceDiagram
participant User
participant "STT Block" as Block
participant "API Route" as API
participant "Audio Extractor" as Extractor
participant "STT Provider" as Provider
User->>Block: "Upload audio/video file"
Block->>API: "POST /api/proxy/stt with file"
API->>API: "Authenticate request"
API->>API: "Download file from storage"
alt Video File
API->>Extractor: "Extract audio from video"
Extractor->>Extractor: "Create temp files with Date.now()"
Extractor->>Extractor: "Convert using ffmpeg"
Extractor->>API: "Return audio buffer"
end
API->>Provider: "Send audio for transcription"
Provider->>API: "Return transcript with segments"
API->>Block: "Return transcript data"
Block->>User: "Display transcript"

Loading

[greptile-apps]

_{26 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}