You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The tutor chat is request/response, not streamed. POST /api/learn/chat (backend/routes/learn.py:552) runs the model to completion and returns a single JSON dict, so the user watches a spinner for the whole multi-second generation instead of seeing tokens as they arrive.
Agent path — _chat_via_agent (learn.py:443) calls await agent.run(user_message, ...) (:499) and returns {reply, graph_update: {}, mastery_changes: []} (graph changes are persisted in-band by apply_graph_update_tool during the run).
Legacy path — _legacy_chat (learn.py:509) calls the blocking call_gemini_multiturn and returns {reply, graph_update, mastery_changes}.
Frontend — sendChat (frontend/src/lib/api.ts:105) is a plain fetchJSON that awaits the full reply; ChatPanel.tsx only renders the assistant bubble once it resolves.
SSE plumbing already exists and is proven on the document-upload path (api.ts:426-448 consuming streamSSE from lib/sse.ts). This issue wires the same mechanism into chat.
Proposal
Add a streaming chat turn over SSE — reuse streamSSE on the client and Pydantic AI's agent.run_stream() on the server. Stream assistant text token-by-token, then emit a terminal event carrying the post-run graph_update/mastery_changes. Keep the existing non-streaming endpoint as a fallback.
Add a streaming entrypoint — either POST /api/learn/chat/stream or content-negotiated on /chat — returning StreamingResponse(media_type="text/event-stream").
Replace await agent.run(...) with async with agent.run_stream(...) as result: and iterate result.stream_text(delta=True), emitting one SSE token event per delta. Reuse the existing run_kwargs (deps, message_history, model override, _build_pro_model_settings()) and the use_shared_context constraint injection from _chat_via_agent.
Preserve message-persistence ordering. Today the agent path persists messages out-of-band in chat() (_load_message_history loads prior turns BEFORE the new turn; the assistant row is written after). The stream must save the assistant message once, after the stream completes — and must NOT persist a partial reply on client disconnect/abort.
Tool-call nuance.chat_tutor registers apply_graph_update_tool, which runs and persists graph changes mid-run. Stream text only for the final model output; after run_stream finishes, emit a terminal done event with {graph_update, mastery_changes}. This is the natural seam to also emit [P2] Live-update Learn progress card via SSE graph deltas #74's graph_update deltas — do them together.
Errors + tracing. On model/tool failure mid-stream, emit a terminal error event (mirror the upload stream); keep _legacy_chat reachable for non-streaming clients. Stamp request_id / X-Request-ID on the stream for trace correlation, matching chat() (:557-563).
Add a streaming sendChatStream(..., { onToken, signal }) that consumes streamSSE<ChatEvent> exactly like the upload path (api.ts:426-448): append each token delta to the in-progress assistant bubble; on done, reconcile {graph_update, mastery_changes} into Learn state.
ChatPanel.tsx: render a live streaming assistant bubble with a typing indicator, finalized on done; disable the composer while streaming; support stop/abort via the AbortControllersignal.
Problem
The tutor chat is request/response, not streamed.
POST /api/learn/chat(backend/routes/learn.py:552) runs the model to completion and returns a single JSON dict, so the user watches a spinner for the whole multi-second generation instead of seeing tokens as they arrive._chat_via_agent(learn.py:443) callsawait agent.run(user_message, ...)(:499) and returns{reply, graph_update: {}, mastery_changes: []}(graph changes are persisted in-band byapply_graph_update_toolduring the run)._legacy_chat(learn.py:509) calls the blockingcall_gemini_multiturnand returns{reply, graph_update, mastery_changes}.sendChat(frontend/src/lib/api.ts:105) is a plainfetchJSONthat awaits the full reply;ChatPanel.tsxonly renders the assistant bubble once it resolves.SSE plumbing already exists and is proven on the document-upload path (
api.ts:426-448consumingstreamSSEfromlib/sse.ts). This issue wires the same mechanism into chat.Proposal
Add a streaming chat turn over SSE — reuse
streamSSEon the client and Pydantic AI'sagent.run_stream()on the server. Stream assistant text token-by-token, then emit a terminal event carrying the post-rungraph_update/mastery_changes. Keep the existing non-streaming endpoint as a fallback.Scope
Backend —
backend/routes/learn.py,backend/agents/chat_tutor.pyPOST /api/learn/chat/streamor content-negotiated on/chat— returningStreamingResponse(media_type="text/event-stream").await agent.run(...)withasync with agent.run_stream(...) as result:and iterateresult.stream_text(delta=True), emitting one SSEtokenevent per delta. Reuse the existingrun_kwargs(deps,message_history, model override,_build_pro_model_settings()) and theuse_shared_contextconstraint injection from_chat_via_agent.chat()(_load_message_historyloads prior turns BEFORE the new turn; the assistant row is written after). The stream must save the assistant message once, after the stream completes — and must NOT persist a partial reply on client disconnect/abort.chat_tutorregistersapply_graph_update_tool, which runs and persists graph changes mid-run. Stream text only for the final model output; afterrun_streamfinishes, emit a terminaldoneevent with{graph_update, mastery_changes}. This is the natural seam to also emit [P2] Live-update Learn progress card via SSE graph deltas #74'sgraph_updatedeltas — do them together.errorevent (mirror the upload stream); keep_legacy_chatreachable for non-streaming clients. Stamprequest_id/X-Request-IDon the stream for trace correlation, matchingchat()(:557-563).Frontend —
frontend/src/lib/api.ts,components/ChatPanel.tsx,screens/Learn.tsxsendChatStream(..., { onToken, signal })that consumesstreamSSE<ChatEvent>exactly like the upload path (api.ts:426-448): append eachtokendelta to the in-progress assistant bubble; ondone, reconcile{graph_update, mastery_changes}into Learn state.ChatPanel.tsx: render a live streaming assistant bubble with a typing indicator, finalized ondone; disable the composer while streaming; support stop/abort via theAbortControllersignal.sendChatforIS_LOCAL_MODEand as a fallback.Non-goals
/actionor/mode-switch(separate follow-ups if wanted).Dependencies / coordination
done/graph_updateevent is shared.Acceptance
graph_update/mastery_changesare applied exactly once ondone.