Uh oh!
There was an error while loading. Please reload this page.
Multimodal web app - #210
Open
seyeong-han wants to merge 10 commits into
Open
Conversation
seyeong-han
commented
Feb 12, 2026
Contributor
| Example | Description |
|---|---|
| ask-anything-app | A full-stack web application enabling real-time camera interaction and chat via Gemma 3 Vision. |
| text-runtime | Local text generation using the Qwen 3 LLM. |
| text-image-runtime | Multimodal vision-language capabilities powered by Gemma 3. |
| voice-runtime | High-efficiency speech-to-text processing using Whisper. |
| object-detection-runtime | Real-time visual perception and tracking with YOLO. |
Glass-styled React frontend with: - Full-screen camera background with frozen frame during processing - Floating glass chat panel with auto-expanding messages - Voice recording and transcription support - CSS glassmorphism effects FastAPI backend with: - Gemma 3 vision-language model integration - Whisper speech-to-text transcription - Health check and status endpoints
ExecuTorch-based YOLO inference with XNNPACK backend: - yolo_test.py for inference with custom image/model/confidence - Support for YOLO26m end-to-end model - Outputs bounding boxes with class labels and confidence scores
Vision-language inference with Gemma3 4B model: - runtime_inference.py for XNNPACK backend inference - run.py for simplified CLI interface - Support for image + text prompt input Model download: hf download lucylq/gemma3 --local-dir models/gemma3
Qwen3-0.6B inference with optimum-executorch: - Chat template support with thinking mode - XNNPACK backend for CPU optimization Model: hf download larryliu0820/Qwen3-0.6B-ExecuTorch-XNNPACK
Speech-to-text with Whisper tiny model: - whisper_inference.py using optimum-executorch - Sample audio: obama_short20.wav (20s speech) Model: hf download larryliu0820/whisper-tiny-ExecuTorch-XNNPACK
kirklandsign
commented
Feb 20, 2026
Contributor
Duplicated multimodal/text-image-runtime/example.jpg Maybe upload to some static storage? (GH can do) Also possible to expose the server as openai compatible api? like /v1/chat/completion |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.