fix(llm,web): chat and completion prompts keep the active model's full context, embedding cold starts and deploy windows stop breaking chat, and API/error responses report real status codes

[WilliamAGH]

Summary

Latency and correctness fixes across the LLM path and web layer: prompts now truncate to the limits of the model actually serving each request (previously every GPT-5-family chat — including gateway gpt-5.4 with 400K context — was silently cut to 7K tokens), live chat turns no longer pay 20–60s serverless embedding cold starts, error and API responses report real status codes, the SPA survives rolling deploys with a guarded auto-reload, and the frontend picks up the grouped Dependabot security bumps.

Changes

Bug Fixes

• GPT-5-family chats through OpenAI direct/the LLM gateway keep their full context: the 7K input cap modeled GitHub Models' 8K tier but was applied family-wide on both the streaming and completion paths, truncating every gpt-5.4 chat turn to 7K tokens; the cap now applies only when GitHub Models serves a GPT-5 model, and the truncation notice no longer claims an 8K limit for a 100K truncation (OpenAiRequestFactory.prepareStreamingRequest/buildCompletionRequest)
• Completion prompts no longer truncate to the unused provider's limit: truncation moved inside buildCompletionRequest, keyed to the active provider's model; fallback re-truncates with the fallback model's own limit (OpenAIStreamingService.complete; fixes[Detail Bug] Prompt truncation uses limits from unused provider, forcing 7K cap on high-context models #40, contributed in Scope completion truncation to active provider #49 by @pntech20)
• o-series models keep their full input window: o1/o3/o3-mini use the default 100K cap instead of inheriting the GPT-5 tier cap (OpenAiRequestFactory)
• Chat turns no longer absorb serverless embedding cold starts: a 4-minute keep-alive probe plus startup warmup keeps the remote embedding model resident (reproduced: 23.7s cold vs 1.0s warm); slow probes log WARN as a cold-start record (EmbeddingModelKeepAlive)
• Missing assets and error pages return their real HTTP status instead of 200: the error-dispatch status now survives the forward to the error documentation pages, so deploy-window asset misses surface as clean 404s instead of browser strict-MIME module errors; directly-browsed docs stay 200 and the swallowed IOException logs (ErrorDocumentationController, CustomErrorController)
• Unknown /api paths return JSON 404 instead of the SPA shell with 200: a (?!api$) guard keeps the API namespace out of the SPA fallback patterns, and bare /api joins the JSON error branch (WebMvcConfig, CustomErrorController)
• The SPA recovers from rolling-deploy chunk failures: a vite:preloadError listener reloads once per session (sessionStorage-guarded) to pick up the fresh build instead of leaving a dead page (frontend/src/main.ts)
• Chrome's deprecated-meta warning silenced: mobile-web-app-capable added alongside the apple variant (frontend/index.html)

Dependencies

• Frontend npm group bump (Dependabot build(deps): bump the npm_and_yarn group across 1 directory with 8 updates #50): dompurify 3.3.1→3.4.0 (markdown sanitizer; verified its FORBID_TAGS/ADD_ATTR changes don't intersect with our single static config), svelte 5.48→5.55, vite 6.4.1→6.4.2, plus transitive security patches; validated with clean install, lint, svelte-check, production build, and 80/80 tests (frontend/package.json, lockfile)

Refactoring

• Removed the two orphaned public truncation overloads left after the provider-scoped fix; regression tests exercise buildCompletionRequest, covering the full provider-to-model-id truncation path (OpenAiRequestFactory, OpenAiRequestFactoryTest)

Breaking Changes

None

Related Issues

Closes#40

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a scheduled EmbeddingModelKeepAlive component to probe embedding providers and moves completion prompt truncation into OpenAiRequestFactory so truncation uses the resolved model’s token limits; also updates a few frontend dependency versions.

Changes

Embedding Model Keep-Alive

Layer / File(s)	Summary
EmbeddingModelKeepAlive component and keep-alive tests src/main/java/com/williamcallahan/javachat/service/EmbeddingModelKeepAlive.java, src/test/java/com/williamcallahan/javachat/service/EmbeddingModelKeepAliveTest.java	Spring @Component schedules keepEmbeddingModelWarm() with fixed warmup/interval/threshold, calls EmbeddingClient.embed(...), logs WARN on provider unavailability without rethrowing, and logs WARN/DEBUG depending on probe latency. Tests verify single embed invocation and exception suppression.

Provider-Aware Prompt Truncation

Layer / File(s)	Summary
Model-aware truncation in OpenAiRequestFactory src/main/java/com/williamcallahan/javachat/service/OpenAiRequestFactory.java, src/test/java/com/williamcallahan/javachat/service/OpenAiRequestFactoryTest.java	buildCompletionRequest resolves the selected modelId and runs truncatePromptForCompletion(prompt, modelId) which applies a GPT-5-family 7K input cap only when the resolved model is GPT-5-family; truncated prompts receive the appropriate notice prefix. Tests confirm behavior for OPENAI, o-series, and GITHUB_MODELS selections.
OpenAIStreamingService removes local truncation src/main/java/com/williamcallahan/javachat/service/OpenAIStreamingService.java	complete(...) no longer computes a local truncated prompt; it passes the original prompt into requestFactory.buildCompletionRequest(...), delegating truncation to the factory.

Frontend dependency updates

Layer / File(s)	Summary
package.json dependency bumps frontend/package.json	Bumps dompurify and upgrades vite (and adjusts svelte span) in frontend dev dependencies.

Sequence Diagram(s)

sequenceDiagram
participant SpringScheduler as Spring Scheduler
participant KeepAlive as EmbeddingModelKeepAlive
participant EmbedClient as EmbeddingClient
SpringScheduler->>KeepAlive: keepEmbeddingModelWarm() (scheduled)
KeepAlive->>KeepAlive: record probe start time
KeepAlive->>EmbedClient: embed(probe text)
alt Provider Available
EmbedClient-->>KeepAlive: embedding result
KeepAlive->>KeepAlive: compare latency to cold-start threshold
KeepAlive->>KeepAlive: log WARN (cold) or DEBUG (warm)
else Provider Unavailable
EmbedClient-->>KeepAlive: EmbeddingServiceUnavailableException
KeepAlive->>KeepAlive: log WARN, suppress exception
KeepAlive-->>SpringScheduler: no exception propagated
end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• WilliamAGH/java-chat#49: Directly related truncation refactor to scope prompt limits by the active provider instead of aggregating limits across configured providers.

Poem

🌱 A tiny probe wakes sleeping models at dawn,
Prompts trimmed by the model they’re truly upon.
Dependencies polished, tests sing their part—
Warm embeddings and kinder truncation, smart! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 26.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	All requirements from issue #40 are met: truncation moved to buildCompletionRequest keyed to active provider's model ID (#40), o-series models use default 100K cap instead of GPT-5's 7K, provider fallback re-truncates with fallback model's limits, orphaned overloads removed, and regression tests added covering the full provider-to-model-id truncation path.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to the linked issue: prompt truncation refactoring, embedding keep-alive implementation, and orphaned method removal. The minor frontend dependency bumps (dompurify, vite) are reasonable maintenance updates unrelated to the core fixes.
Title check	✅ Passed	The title comprehensively captures the main changes: prompt context preservation for active models, embedding cold start fixes, and HTTP status code corrections.
Description check	✅ Passed	The description is well-structured and directly relates to the changeset, providing clear summaries of bug fixes, dependencies, and refactoring across multiple files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch dev

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:b6878a450f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

This PR fixes two LLM-path latency/correctness issues: it scopes completion prompt truncation to the active provider/model (so high-context models don’t get capped by an unused provider’s GPT‑5 defaults), and it adds an embedding keep-alive scheduler to prevent chat turns from stalling on embedding cold starts.

Changes:

• Move completion truncation into OpenAiRequestFactory.buildCompletionRequest(...), keyed to the resolved model ID for each provider attempt (including fallback).
• Adjust truncation rules so GPT‑5-family models use the 7K safe input budget while o‑series models no longer inherit that cap.
• Add EmbeddingModelKeepAlive + tests to probe the embedding provider on a cadence and avoid cold-start latency on user requests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/main/java/com/williamcallahan/javachat/service/OpenAIStreamingService.java	Stops pre-truncating before provider selection; passes raw prompt so each provider attempt can truncate to its own model limit.
src/main/java/com/williamcallahan/javachat/service/OpenAiRequestFactory.java	Applies completion prompt truncation inside buildCompletionRequest using the resolved model ID; adjusts GPT‑5 vs non‑GPT‑5 token budgeting.
src/main/java/com/williamcallahan/javachat/service/EmbeddingModelKeepAlive.java	Adds scheduled embedding probe to keep serverless/self-hosted embedding models warm and avoid cold-start latency.
src/test/java/com/williamcallahan/javachat/service/OpenAiRequestFactoryTest.java	Adds regression tests ensuring provider/model-specific truncation behavior (OpenAI vs GitHub Models; GPT‑5 vs o‑series).
src/test/java/com/williamcallahan/javachat/service/EmbeddingModelKeepAliveTest.java	Adds unit tests verifying probing behavior and that provider unavailability is not propagated.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.