feat(local-ai): plug-and-play Local AI settings — capability, downloads, storage, fallback by qnbs · Pull Request #138 · qnbs/WorldScript-Studio

qnbs · 2026-06-14T18:37:15Z

User description

Why

Local AI is the highest-leverage adoption gap: all the on-device plumbing exists (WebLLM worker pool, device profiling, progress emitter, engine cache) but it was invisible to users — models downloaded lazily with no warning, no way to see device capability, no way to reclaim the multi-GB on-disk cache, and the fallback chain was undocumented. This makes local inference legible and controllable.

What

A dedicated Settings → AI Models → Local AI section:

Capability card — WebGPU availability, device-class badge, recommended model, first-use onboarding hint
Model manager — per-model Download with storage-size warnings; session Ready badge. Reuses the worker-path generateLocalText → inferenceProgressEmitter → LocalAiDownloadProgress modal, so the download UX is identical to a real inference (GPU mutex + cancel + ETA)
Storage card — usage estimate + Clear Local Models
Fallback-chain explainer — WebGPU → WASM (ONNX) → Transformers.js → heuristic
Throughput indicator — last-run tokens/sec

New logic (everything else is reuse)

services/ai/localModelStorageService.ts — estimateLocalModelStorage() + clearLocalModels(). The @domain/ai-core release fns only free in-memory GPU/WASM handles; the downloaded weights live in the Cache API. This estimates and clears that on-disk footprint.
preloadLocalModel() wrapper in services/localAiFacade.ts (routes through the worker path; records best-effort tok/s).

Lockstep

Wired into SettingsView nav + search hints + guide; help article made actionable (tryActionId).
Docs: docs/LOCAL-AI.md (setup & troubleshooting) + README hub link.
i18n: 39 keys × 11 locales (parity + placeholder guard green).
Tests: localModelStorageService (7) + LocalAiSection (8) — all green.

Verification (local, sequential — low-end hardware)

pnpm run lint ✅ · pnpm run typecheck ✅ · pnpm run i18n:check ✅
New tests: 15 passed; affected suites (helpCatalogIntegrity, helpDocRetrieval, localAiFacade, i18nPlaceholders) ✅

Out of scope (explicit follow-ups, separate PRs)

Voice benchmarks/GPU-tracing · full data-portability/backup UX · PWA offline-after-download hardening · app-wide error-boundary/logging sweep · AI-provider cost/usage + quota auto-switch.

🤖 Generated with Claude Code

CodeAnt-AI Description

Add a Local AI settings page for downloading, managing, and troubleshooting on-device models

What Changed

Added a new Local AI section in Settings with WebGPU status, device class, recommended model, download actions, storage usage, and a visible Ready state for downloaded models
Users can clear downloaded local models from the same page, with a confirmation dialog and a live warning when storage is too low for a model
The page now explains the offline fallback chain and shows the last measured local speed in tokens per second
Download cancel now stops the active preload instead of only closing the dialog, and opening Local AI is now reachable from Settings search and the help guide
Added setup/troubleshooting documentation and updated translations for the new Local AI screens

Impact

✅ Easier offline AI setup
✅ Fewer failed model downloads
✅ Faster recovery of disk space

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

…ds, storage, fallback Adds a dedicated Settings → AI Models → Local AI section that makes on-device inference legible and controllable (the highest-leverage local-AI UX gap): - Capability card: WebGPU availability, device-class badge, recommended model, first-use onboarding hint - Model manager: per-model Download (reuses worker-path generateLocalText → inferenceProgressEmitter → LocalAiDownloadProgress modal) with storage-size warnings; session "Ready" badge - Storage card: usage estimate + "Clear Local Models" (new on-disk cache mgmt) - Fallback-chain explainer (WebGPU → WASM → Transformers.js → heuristic) - Last-run throughput (tok/s) indicator New: services/ai/localModelStorageService.ts (estimate/clear Cache API model weights — the @domain/ai-core release fns only free in-memory handles), and a preloadLocalModel() wrapper in localAiFacade. Wired into SettingsView nav + search hints + guide; help article made actionable; docs/LOCAL-AI.md + README hub link. i18n: 39 keys × 11 locales. Tests: service (7) + component (8). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

codeant-ai · 2026-06-14T18:37:18Z

CodeAnt AI is reviewing your PR.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

vercel · 2026-06-14T18:37:20Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
storycraft-studio	Ready	Preview, Comment	Jun 14, 2026 8:34pm

qnbs · 2026-06-14T18:37:28Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T18:37:33Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T18:39:12Z

Sequence Diagram

This diagram shows how the new Local AI Settings section lets users preload on-device models via the existing worker path and manage their downloaded storage, including clearing cached weights.

sequenceDiagram
participant User
participant SettingsUI as Settings UI
participant LocalAI as Local AI backend
User->>SettingsUI: Open Local AI section
SettingsUI->>LocalAI: Load capability, cached models, storage, throughput
LocalAI-->>SettingsUI: Capability report, model list, storage estimate, last tokens/sec
User->>SettingsUI: Click Download for model
SettingsUI->>LocalAI: Preload model via worker (download and warm up)
LocalAI-->>SettingsUI: Model ready, updated storage, new throughput sample
User->>SettingsUI: Click Clear Local Models
SettingsUI->>LocalAI: Clear downloaded model caches and release engines
LocalAI-->>SettingsUI: Cleared cache count and refreshed storage/model state

Generated by CodeAnt AI

codeant-ai · 2026-06-14T18:43:26Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T18:43:51Z

CodeAnt AI finished reviewing your PR.

…ph, deterministic tests - Replace inline alertdialog confirm with the shared Modal (focus trap + restore + Escape + alertdialog semantics), matching the factory-reset pattern (#138 threads 3+4) - Replace hardcoded '—' device-class fallback with t('settings.ai.localAi.detecting') (thread 2) - Make localModelStorageService tests deterministic by explicitly stubbing absent caches/navigator.storage instead of relying on runtime env defaults (thread 1) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

qnbs · 2026-06-14T18:48:23Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T18:48:27Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T18:50:17Z

Sequence Diagram

This diagram shows how the new Local AI settings let users explicitly download and warm local models via the existing worker path, record throughput, and manage on-disk model storage through a cache-clearing service.

sequenceDiagram
participant User
participant LocalAISettings
participant LocalAiFacade
participant LocalModelStorage
User->>LocalAISettings: Click Download for a model
LocalAISettings->>LocalAiFacade: Preload selected model via worker path
LocalAiFacade->>LocalAiFacade: Run minimal local generation and measure tokens per second
LocalAiFacade-->>LocalAISettings: Mark model ready with throughput sample
LocalAISettings->>LocalModelStorage: Refresh local model storage estimate
LocalAISettings-->>User: Show Ready badge and storage usage
User->>LocalAISettings: Click Clear Local Models
LocalAISettings->>LocalModelStorage: Release engines and delete local model caches
LocalModelStorage-->>LocalAISettings: Cleared cache count and updated usage
LocalAISettings-->>User: Confirm cleared models and updated storage

Generated by CodeAnt AI

codeant-ai · 2026-06-14T18:56:16Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

…bLLM readiness - localModelStorageService: quota/usage/free/percent are now nullable; when the StorageManager has no usable quota they stay null (estimateAvailable=false) instead of 0, so the UI no longer mislabels every model as too large (#138) - await the async releaseAllOnnxSessions() in clearLocalModels so failures are caught and session release completes before cache deletion - preloadLocalModel returns { layer, modelId, downloaded }; downloaded is true ONLY for a verified WebLLM warm of the requested model (not an ONNX/Transformers fallback or the multi-tab lock message). Throughput recorded only when downloaded - LocalAiSection: mark a model "Ready" / announce only when downloaded===true; drop main-thread-only listCachedWebLlmEngines seeding (didn't reflect the worker path); reset readyIds on clear; size warning + usage bar gated on known quota - Tests updated + new no-WebGPU-fallback case (no false Ready badge) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

qnbs · 2026-06-14T19:04:10Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T19:04:14Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T19:05:23Z

Sequence Diagram

This PR adds a Local AI settings section where users can preload on-device models via the existing worker-based local inference path and manage their on-disk model storage by clearing cached weights.

sequenceDiagram
participant User
participant SettingsUI
participant LocalAiService
participant LocalAiRuntime
User->>SettingsUI: Open Local AI settings
SettingsUI->>LocalAiService: Load device capability and storage estimate
LocalAiService-->>SettingsUI: Capability and storage info
User->>SettingsUI: Download selected local model
SettingsUI->>LocalAiRuntime: Preload model via worker path
LocalAiRuntime-->>SettingsUI: Model downloaded and throughput sample
SettingsUI-->>User: Show model Ready and tokens per second
User->>SettingsUI: Clear Local Models
SettingsUI->>LocalAiService: Release engines and delete model caches
LocalAiService-->>SettingsUI: Cleared count and refreshed storage info

Generated by CodeAnt AI

codeant-ai · 2026-06-14T20:12:16Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

…odel labels - handleDownload now catches errors (the click handler voids the promise), logs via StructuredLogger, and announces settings.ai.localAi.downloadFailed instead of leaking an unhandled rejection; handleClear hardened the same way (clearFailed) - Model display names rendered via i18n keys (settings.ai.localAi.modelLabel.*) with the raw @domain/ai-core label as fallback — list, recommendation, and ready announcement - 9 new keys × 11 locales; new test for the download-failure path Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

qnbs · 2026-06-14T20:16:59Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T20:17:04Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T20:18:23Z

Sequence Diagram

This PR adds a Local AI settings section where users can pre-download on-device models through the existing worker pipeline, see device capability and throughput, and safely clear cached model weights with a global busy check.

sequenceDiagram
participant User
participant Settings
participant LocalAI
User->>Settings: Open Local AI settings
Settings->>LocalAI: Fetch capability, ready models, storage estimate, throughput sample
LocalAI-->>Settings: Return capability and storage info
User->>Settings: Click Download for model
Settings->>LocalAI: Preload model via worker pipeline
LocalAI-->>Settings: Model warmed, update ready state and throughput, refresh storage
User->>Settings: Click Clear Local Models
Settings->>LocalAI: Check if any local AI work is in flight
alt Local AI idle
Settings->>LocalAI: Release engines and delete cached model data
LocalAI-->>Settings: Cleared cache count and new storage estimate
else Local AI busy
LocalAI-->>Settings: Report busy, do not clear
end

Generated by CodeAnt AI

codeant-ai · 2026-06-14T20:23:34Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

activePreloadAbort was cleared unconditionally in finally, so an older preload finishing could erase a newer preload's cancel hook (cancel/abort would silently no-op). Capture our own abortHook and only clear it if it's still the active one (identity guard). New facade test proves a newer preload stays cancellable after an older one resolves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

qnbs · 2026-06-14T20:25:49Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T20:25:54Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T20:27:11Z

Sequence Diagram

This diagram shows how the new Local AI settings section lets a user preload a local model: the Settings UI calls the local AI facade to download and warm the model, records throughput, refreshes storage estimates, and marks the model ready for offline use.

sequenceDiagram
participant User
participant Settings as Local AI settings
participant Facade as Local AI facade
participant Engine as Local inference engine
participant Storage as Local model storage
User->>Settings: Click Download for a local model
Settings->>Facade: Preload selected model for offline use
Facade->>Engine: Download weights and run short warmup generation
Engine-->>Facade: Return layer used and generated text
Facade-->>Settings: Report model downloaded and tokens per second
Settings->>Storage: Estimate local model storage usage
Storage-->>Settings: Usage, quota, and cache count
Settings-->>User: Show Ready badge, updated storage, and throughput indicator

Generated by CodeAnt AI

codeant-ai · 2026-06-14T20:30:58Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

…o overlapping preloads) The download buttons ignored externalLoading, letting a user start a preload while a WebLLM download ran elsewhere — overlapping local-AI work that made the single global cancel hook ambiguous. Add externalLoading to the per-model disabled guard. New test asserts downloads disable when the progress emitter reports "loading". (The duplicate cancel-hook race finding in this wave was already fixed in 2042cae.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

qnbs · 2026-06-14T20:33:59Z

@CodeAnt-AI review

codeant-ai · 2026-06-14T20:34:03Z

CodeAnt AI is running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-ai · 2026-06-14T20:35:36Z

Sequence Diagram

This diagram shows how the new Local AI settings page preloads a local model through the shared worker pipeline and how it safely clears downloaded model storage when requested.

sequenceDiagram
participant User
participant LocalAISettings as Local AI Settings
participant LocalAIFacade as Local AI Facade
participant LocalAIEngine as Local AI Engine
participant StorageService as Local Model Storage Service
User->>LocalAISettings: Open Local AI section and choose model
LocalAISettings->>LocalAIFacade: Preload selected model
LocalAIFacade->>LocalAIEngine: Download and warm model via worker chain
LocalAIEngine-->>LocalAIFacade: Model ready with throughput sample
LocalAIFacade-->>LocalAISettings: Mark model Ready and update throughput
User->>LocalAISettings: Click Clear Local Models
LocalAISettings->>LocalAIFacade: Check if any local AI work is in flight
LocalAISettings->>StorageService: Clear local models (release engines and caches)
StorageService-->>LocalAISettings: Cleared count and refreshed storage estimate

Generated by CodeAnt AI

codeant-ai · 2026-06-14T20:38:15Z

CodeAnt AI finished running the review.

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

codeant-aiBot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Jun 14, 2026

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadtests/unit/ai/localModelStorageService.test.ts

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated
Comment threadcomponents/settings/LocalAiSection.tsx Outdated

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadservices/ai/localModelStorageService.ts Outdated
Comment threadservices/ai/localModelStorageService.ts

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated
Comment threadcomponents/settings/LocalAiSection.tsx

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated
Comment threadservices/localAiFacade.ts Outdated

codeant-aiBot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 14, 2026

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadservices/ai/localModelStorageService.ts Outdated

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx
Comment threadcomponents/settings/LocalAiSection.tsx

codeant-aiBot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 14, 2026

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadservices/localAiFacade.ts

codeant-aiBot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 14, 2026

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadservices/localAiFacade.ts Outdated

codeant-aiBot reviewed Jun 14, 2026
View reviewed changes

Comment threadcomponents/settings/LocalAiSection.tsx Outdated

codeant-aiBot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 14, 2026

qnbs merged commit 478d68a into mainJun 14, 2026
17 checks passed

qnbs deleted the feat/local-ai-ux branch June 14, 2026 20:59

This was referenced Aug 12, 2026
feat: add durable encryption migration journal #337
Merged
feat: production wiring for encryption disable/rotate + recovery UX (Phase 4, #338 items 1+2) #342
Merged

Conversation

qnbs commented Jun 14, 2026• edited by codeant-ai BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

Why

What

New logic (everything else is reuse)

Lockstep

Verification (local, sequential — low-end hardware)

Out of scope (explicit follow-ups, separate PRs)

CodeAnt-AI Description

What Changed

Impact

Checking Your Pull Request

Talking to CodeAnt AI

Example

Preserve Org Learnings with CodeAnt

Example

Retrigger review

Check Your Repository Health

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

vercelBot commented Jun 14, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qnbs commented Jun 14, 2026

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codeant-aiBot commented Jun 14, 2026

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

codeant-aiBot commented Jun 14, 2026

Uh oh!

qnbs commented Jun 14, 2026

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

codeant-aiBot commented Jun 14, 2026

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

qnbs commented Jun 14, 2026

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

codeant-aiBot commented Jun 14, 2026

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

codeant-aiBot commented Jun 14, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

qnbs commented Jun 14, 2026

Uh oh!

codeant-aiBot commented Jun 14, 2026

qnbs commented Jun 14, 2026•
edited by codeant-ai Bot
Loading

vercelBot commented Jun 14, 2026•
edited
Loading