From 229ab29b515d21ec373e0a82d45ca1bc66ddffbc Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Wed, 24 Jun 2026 13:16:27 +0200 Subject: [PATCH] feat(agent): cancellation, timeouts, typed errors, and lifecycle helpers Hardens CoderAgent based on external integration feedback (durable deep-research fan-out on Vercel Workflow), then a quality + correctness pass. - abortSignal now interrupts the *server* run (not just the local socket); a stream cancel() interrupts too, so wedged turns stop holding their workspace. - requestTimeoutMs: per-segment time budget that interrupts and rejects with a retryable CoderChatError (kind: "timeout") instead of hanging. - A stream that ends before the turn settles now rejects with a retryable CoderChatError (kind: "stream_closed") rather than reporting a truncated result as a clean stop; transport errors and server error events are classified into typed, retryable errors too. - CoderAgent.listModels() to discover model hints. - await using support via Symbol.asyncDispose (interrupt + archive on scope exit). - Warn when responseFormat/JSON-schema output is requested (unsupported server-side; use @coder/ai-sdk-provider + generateObject instead). - Docs: Agent-vs-provider decision guide, error handling, workspaces/quota, durable-workflow recipe, and timeout semantics. Change-Id: I19e900ea5a4e8d7c8ff34813ae5f47399dc5d28d Co-Authored-By: Claude Opus 4.8 (1M context) Signed-off-by: Thomas Kosiewski --- README.md | 6 + packages/agent/README.md | 175 ++++++++++++-- packages/agent/src/agent/coder-agent.ts | 53 +++++ packages/agent/src/model/language-model.ts | 176 ++++++++++++-- packages/agent/test/unit/agent.test.ts | 257 +++++++++++++++++++-- packages/provider/README.md | 10 + 6 files changed, 623 insertions(+), 54 deletions(-) diff --git a/README.md b/README.md index 15ac687..2799500 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,12 @@ install instructions, usage, and API docs. | [`@coder/ai-sdk-agent`](./packages/agent) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-agent.svg)](https://www.npmjs.com/package/@coder/ai-sdk-agent) | A Vercel AI SDK–compliant **`Agent`** (AI SDK v6) backed by **Coder Agents**, Coder's server-side agent runtime. `new CoderAgent()` returns a real `Agent` — `generate()`, `stream()`, tool calls, the whole interface. | | [`@coder/ai-sdk-provider`](./packages/provider) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-provider.svg)](https://www.npmjs.com/package/@coder/ai-sdk-provider) | A **Vercel AI SDK provider** that routes `generateText` / `streamText` calls through your Coder deployment's [AI Gateway](https://coder.com/docs/ai-coder/ai-gateway). Point it at your deployment with a Coder API token and use any model it proxies — no raw provider keys, with per-user auth and audit. | +**Which package?** Need a **model** (text, streaming, or schema‑constrained +structured output) through your deployment → `@coder/ai-sdk-provider`. Need Coder's +**server‑side agent** (multi‑step tool loop, MCP, workspace file/shell tools) → +`@coder/ai-sdk-agent`. Need to run a **CLI coding agent** (Claude Code, Codex) +inside a workspace → `@coder/ai-sdk-sandbox`. + ## Contributing Development setup, the command reference, and how releases work all live in diff --git a/packages/agent/README.md b/packages/agent/README.md index 8ec0598..0271845 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -21,6 +21,25 @@ context compaction. The Vercel AI SDK runs its loop **client‑side**. This pack bridges the two, so a Coder agent looks and feels like a native AI SDK agent without re‑implementing the loop. +## Agent vs. provider — which package? + +Two packages, two jobs: + +- **`@coder/ai-sdk-agent` (this package)** — Coder's **server‑side agent**: the + multi‑step tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell + tools, sub‑agents, and compaction all run on the deployment. Each `CoderAgent` + is one server chat ("session") and may provision a workspace. Reach for it when + you need **server‑side tools, MCP, or a workspace**. +- **[`@coder/ai-sdk-provider`](../provider)** — **plain model calls** through + Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, and + **`generateObject` for schema‑constrained structured output**. No chat, no + workspace, natively cancelable. Reach for it when you just need **a model** + (plan / extract / summarize / classify) with no server‑side tools. + +Rule of thumb: **need server‑side tools, MCP, or a workspace → Agent; need a model +→ provider.** They compose — a multi‑step pipeline often uses the provider for its +pure text/JSON steps and the Agent only for the steps that touch tools. + ## Install ```bash @@ -182,31 +201,131 @@ One `CoderAgent` instance maps to one chat ("session") on the Coder server. The created on the first turn and reused for subsequent `generate()`/`stream()` calls (multi‑turn conversation with server‑side history). `agent.chatId` is the current chat id. -- `agent.resetSession()` — start a fresh chat on the next turn. +- `agent.resetSession()` — start a fresh chat on the next turn (reuse one instance for sequential turns; you don't need a new agent per turn). - `agent.interrupt()` — interrupt an in‑flight generation. -- `agent.archive()` — archive the underlying chat (cleanup). +- `agent.archive()` — archive the underlying chat (cleanup; see [Cleanup](#cleanup)). +- `agent.listModels()` — list the deployment's model configs, so you don't have to guess the `model` hint. - Resume a prior chat: `new CoderAgent({ …, chatId: "…" })`. -A single instance is **single‑flight** — don't run concurrent generations against it. +A single instance is **single‑flight** — don't run concurrent generations against it. For concurrency, use one instance per session (and see [Workspaces & quota](#workspaces--quota)). + +## Timeouts & cancellation + +Pass an `abortSignal` to `generate()`/`stream()` to cancel a turn. Aborting +**interrupts the server‑side run** (not just the local socket), so the chat stops +generating and releases its resources instead of running on, orphaned. Tearing +down a `stream()` early (cancelling the stream) interrupts the run too. + +For a hard ceiling, set `requestTimeoutMs`. If a segment runs longer (e.g. the +server is wedged, or a workspace can't be scheduled), the run is interrupted and +the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) instead of +hanging: + +```ts +const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 }); +``` + +`requestTimeoutMs` bounds **each server segment** — one model round‑trip until it +settles or pauses for a client tool. A multi‑step `generate()` that drives client +tools runs several segments, so it bounds each one, not the whole call. To cap the +**total** wall‑clock of a multi‑step call, pass a deadline as the signal instead: + +```ts +await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) }); +``` + +If the event stream drops before the turn settles, the call rejects with +`CoderChatError` (`kind: "stream_closed"`, retryable) rather than returning a +truncated result as if the turn had finished. + +## Cleanup + +`archive()` soft‑hides the chat (it stays in listings as `archived: true`; there +is no hard delete yet). To make cleanup ride scope exit instead of a `finally` you +have to remember, the agent is an **async disposable**: + +```ts +await using agent = new CoderAgent({ + /* … */ +}); +const { text } = await agent.generate({ prompt: "…" }); +// agent.archive() runs automatically when the scope exits (best‑effort). +``` + +In a request handler that returns before a fire‑and‑forget `archive()` settles, the +archive can be abandoned — `await using` (or an awaited `archive()` in `finally`) +avoids accumulating live chats. + +## Handling errors + +All errors extend `CoderAgentError`. Two carry structured detail you can branch on: + +- **`CoderApiError`** — an HTTP request failed. Fields: `status`, `method`, `path`, `detail`. +- **`CoderChatError`** — a turn ended in an error, timed out, or lost its stream. Fields: `kind`, `retryable`, `statusCode`, `provider`. + +```ts +import { CoderApiError, CoderChatError } from "@coder/ai-sdk-agent"; + +try { + await agent.generate({ prompt: "…" }); +} catch (err) { + if (err instanceof CoderChatError && err.retryable) { + // transient (timeout, stream_closed, an upstream 5xx) — back off and retry + } else if (err instanceof CoderApiError && err.status === 429) { + // rate limited + } else { + throw err; + } +} +``` + +`maxRetries` defaults to `0`: this agent owns server‑side chat state, so an +SDK‑level retry could duplicate a turn. Prefer catching `retryable` errors and +retrying the whole step deliberately. + +## Structured output + +`CoderAgent` does **not** constrain output to a JSON schema — chatd has no +server‑side `response_format`, so a `responseFormat` / `experimental_output` +request emits a warning and is best‑effort at most. For reliable +schema‑constrained output, use **[`@coder/ai-sdk-provider`](../provider)** with +`generateObject` / `Output.object` (requires AI Gateway enabled on the +deployment). Use the Agent for the steps that need server‑side tools; use the +provider for pure text‑in / JSON‑out steps. + +## Workspaces & quota + +A `CoderAgent` is one server‑side chat, and — depending on its configuration and +the deployment — a chat may provision a **Coder workspace** to run its tools. A +deployment caps how many workspaces an account may run at once, so **N agents +running concurrently can need N free workspace slots.** Past the cap, a turn can +sit unscheduled and never settle. This is the most important operational fact when +running many agents at once: + +- Keep your own concurrency below the deployment's workspace limit. +- Set `requestTimeoutMs` so an unschedulable turn fails loudly instead of hanging. +- `archive()` / `await using` each agent so finished chats stop holding resources. +- For steps that don't need server‑side tools, prefer the provider — it never touches a workspace. ## Configuration `CoderAgentSettings`: -| field | description | -| --------------------------------- | ----------------------------------------------------------------------------- | -| `client` \| (`baseUrl` + `token`) | connection (one or the other) | -| `organizationId` | org UUID that owns the chat (required) | -| `model` | model hint: UUID, `provider:model`, model id, or display‑name substring | -| `instructions` | system prompt | -| `tools` | AI SDK `ToolSet` (client‑executed) | -| `workspaceId` | bind the chat to a Coder workspace (enables workspace‑scoped tools) | -| `workspaceFiles` | adapter enabling `uploadToWorkspace()` (write files to the workspace FS) | -| `mcpServerIds` | server‑side MCP servers to enable | -| `planMode` | enable plan mode (`"plan"`) | -| `stopWhen` | AI SDK stop condition(s); default `stepCountIs(64)` | -| `maxRetries` | default `0` — SDK retries can duplicate server‑side turns; override with care | -| `chatId` | resume an existing chat | +| field | description | +| --------------------------------- | ------------------------------------------------------------------------------------------------ | +| `client` \| (`baseUrl` + `token`) | connection (one or the other) | +| `organizationId` | org UUID that owns the chat (required) | +| `model` | model hint: UUID, `provider:model`, model id, or display‑name substring | +| `instructions` | system prompt | +| `tools` | AI SDK `ToolSet` (client‑executed) | +| `workspaceId` | bind the chat to a Coder workspace (enables workspace‑scoped tools) | +| `workspaceFiles` | adapter enabling `uploadToWorkspace()` (write files to the workspace FS) | +| `mcpServerIds` | server‑side MCP servers to enable | +| `planMode` | enable plan mode (`"plan"`) | +| `stopWhen` | AI SDK stop condition(s); default `stepCountIs(64)` | +| `maxRetries` | default `0` — SDK retries can duplicate server‑side turns; override with care | +| `requestTimeoutMs` | per‑turn time budget (ms); interrupts the run and rejects (`kind: "timeout"`) instead of hanging | +| `chatId` | resume an existing chat | ## How it works @@ -224,6 +343,28 @@ CoderAgent (implements ai.Agent) - Streaming text is emitted from `message_part` deltas; fast turns that only produce a full `message` snapshot are diffed against an emitted‑length cursor — so neither double‑counts. +## Durable workflows (Vercel Workflow, step functions, …) + +`CoderAgent` talks to Coder over its own REST + WebSocket client, so it can't ride +a `fetch`‑shim durability layer — each turn must run **inside** a durable step. A +few rules keep it well‑behaved across replays: + +- **One turn per step.** Create the agent, run a single `generate()` (not + `stream()`, so the checkpointed value is the finished result), return. +- **Don't persist the instance across steps.** Persist `agent.chatId` (a string) + and resume with `new CoderAgent({ …, chatId })` in the next step. Never persist + or log the token — read it from the environment in each step. +- **Clean up in the step.** `await using` the agent (or `await agent.archive()` in + a `finally`) so a step that returns early doesn't abandon the chat. +- **Bound each step.** Set `requestTimeoutMs` so a wedged turn fails the step (and + lets the workflow retry) instead of hanging the whole run. +- **Mind concurrency vs. workspaces.** Keep fan‑out width under the deployment's + workspace cap — see [Workspaces & quota](#workspaces--quota). +- **Use the provider for pure steps.** Steps that don't need server‑side tools + (plan / extract / synthesize) are cheaper and natively structured through + [`@coder/ai-sdk-provider`](../provider) + `generateObject` — no chat, no + workspace, no cleanup. + ## Testing ```bash diff --git a/packages/agent/src/agent/coder-agent.ts b/packages/agent/src/agent/coder-agent.ts index c32dced..19abcf7 100644 --- a/packages/agent/src/agent/coder-agent.ts +++ b/packages/agent/src/agent/coder-agent.ts @@ -14,6 +14,7 @@ import { type CoderChatClientOptions, type UploadedChatFile, } from "../coder/client.js"; +import type { ChatModelConfig } from "../coder/types.js"; import { CoderAgentError } from "../errors.js"; import type { FileContent } from "../files.js"; import { CoderLanguageModel } from "../model/language-model.js"; @@ -100,6 +101,18 @@ export interface CoderAgentSettings { * SDK retries could duplicate a turn. Override with care. */ maxRetries?: number; + /** + * Per-segment time budget in milliseconds, applied to each model round-trip + * (chat creation / message / tool-result submission plus the server run until + * it settles). If a segment runs longer (e.g. the server is wedged or a + * workspace can't be scheduled), the run is interrupted server-side and the + * call rejects with a retryable {@link CoderChatError} (`kind: "timeout"`) + * instead of hanging. A multi-step `generate()` driving client tools makes + * several segments, so this bounds each one — to cap total wall-clock for the + * whole call, pass `abortSignal: AbortSignal.timeout(ms)`. Unset or + * non-positive means no limit. + */ + requestTimeoutMs?: number; } /** @@ -162,6 +175,7 @@ export class CoderAgent implements Agent({ @@ -207,6 +221,15 @@ export class CoderAgent implements Agent { + return this.#client.listModelConfigs(signal); + } + /** Start a fresh chatd chat on the next turn. */ resetSession(): void { this.#model.resetSession(); @@ -224,6 +247,36 @@ export class CoderAgent implements Agent { + // archive() only soft-hides the chat; it does not stop a generation. If the + // scope exits mid-turn (e.g. generate() threw, or an early return), interrupt + // first so chatd stops generating and releases the workspace, then archive. + try { + await this.interrupt(); + } catch { + /* best-effort cleanup */ + } + try { + await this.archive(); + } catch { + /* best-effort cleanup */ + } + } + // --- files ---------------------------------------------------------------- /** diff --git a/packages/agent/src/model/language-model.ts b/packages/agent/src/model/language-model.ts index cdfadd7..9ea9183 100644 --- a/packages/agent/src/model/language-model.ts +++ b/packages/agent/src/model/language-model.ts @@ -6,8 +6,9 @@ import type { LanguageModelV3StreamPart, LanguageModelV3StreamResult, LanguageModelV3Usage, + SharedV3Warning, } from "@ai-sdk/provider"; -import { CoderAgentError, CoderChatError } from "../errors.js"; +import { CoderAgentError, CoderApiError, CoderChatError } from "../errors.js"; import { CoderChatClient } from "../coder/client.js"; import type { ChatInputPart, CreateChatRequest } from "../coder/types.js"; import { dataContentToFileContent } from "../files.js"; @@ -44,6 +45,17 @@ export interface CoderLanguageModelConfig { planMode?: "" | "plan"; /** Resume an existing chat instead of creating a new one. */ chatId?: string; + /** + * Per-segment time budget in milliseconds, applied to each model round-trip + * (one `doStream`/`doGenerate` call: chat creation or message/tool-result + * submission, plus the server-side run until it settles or pauses for a client + * tool). If exceeded, the run is interrupted server-side and the call rejects + * with a retryable {@link CoderChatError} (`kind: "timeout"`). A multi-step + * `generate()` that drives client tools makes several segments, so this bounds + * each segment, not the whole call — to cap total wall-clock, pass + * `abortSignal: AbortSignal.timeout(ms)`. Unset or non-positive means no limit. + */ + requestTimeoutMs?: number; } /** @@ -135,9 +147,77 @@ export class CoderLanguageModel implements LanguageModelV3 { ); } this.#inFlight = true; + + // Combine the caller's abort signal with an optional per-turn timeout into a + // single signal (the platform composes and cleans these up for us). When + // there's neither a caller signal nor a timeout, `signal` stays undefined and + // there is no per-turn setup. Keep a reference to our *own* timeout signal so + // a timeout stays distinguishable from a caller abort — even when the caller's + // own signal is itself an `AbortSignal.timeout` (whose reason is a TimeoutError + // too, so reason-sniffing alone would misclassify it). + const externalSignal = options.abortSignal; + const timeoutMs = this.#config.requestTimeoutMs; + const timeoutSignal = + timeoutMs !== undefined && timeoutMs > 0 ? AbortSignal.timeout(timeoutMs) : undefined; + const sources = [externalSignal, timeoutSignal].filter( + (s): s is AbortSignal => s !== undefined, + ); + const signal: AbortSignal | undefined = + sources.length > 0 ? AbortSignal.any(sources) : undefined; + + // Translator is hoisted so `finally` can read whether the turn settled. + const translator = new TurnTranslator({ dynamicToolNames: dynamicToolNames(options.tools) }); + + // Interrupting the *server* run (not just closing the WebSocket) is what frees + // the chat's workspace/resources. Fire it at most once — on abort/timeout, and + // on teardown of an unsettled turn (see `finally`), which also covers stream + // `cancel()` and premature close, neither of which aborts the signal. A chat + // whose id we never received (createChat aborted mid-flight) can't be reached. + let interruptSent = false; + const interrupt = (): void => { + const id = this.#chatId; + if (interruptSent || !id) return; + interruptSent = true; + void this.#config.client.interruptChat(id).catch(() => {}); + }; + signal?.addEventListener("abort", interrupt, { once: true }); + + // Map an abort of the combined signal to the right error. Our own + // `timeoutSignal` having fired means a per-turn timeout; otherwise it's a + // caller abort, re-thrown as the caller's reason (preserving AbortError so the + // AI SDK still recognizes the cancellation). + const throwIfAborted = (): void => { + if (timeoutSignal?.aborted) { + throw new CoderChatError({ + message: `Coder Agent turn exceeded its ${timeoutMs}ms requestTimeoutMs budget.`, + kind: "timeout", + retryable: true, + }); + } + if (signal?.aborted) { + throw ( + externalSignal?.reason ?? new DOMException("The operation was aborted.", "AbortError") + ); + } + }; + try { - const { prompt, abortSignal: signal } = options; - yield { type: "stream-start", warnings: [] }; + const { prompt } = options; + + // chatd does not constrain output to a JSON schema server-side, so a + // `responseFormat: json` request can't be honored. Warn rather than + // silently mislead — schema-constrained output should go through the + // provider (@coder/ai-sdk-provider) instead. + const warnings: SharedV3Warning[] = []; + if (options.responseFormat?.type === "json") { + warnings.push({ + type: "unsupported", + feature: "responseFormat", + details: + "Coder Agents does not enforce a JSON schema server-side, so structured output is best-effort (not schema-constrained). For reliable structured output, use @coder/ai-sdk-provider (createCoder) with generateObject / Output.object.", + }); + } + yield { type: "stream-start", warnings }; const action = classifyTurnAction(prompt); if (action.kind === "noop") { @@ -146,7 +226,6 @@ export class CoderLanguageModel implements LanguageModelV3 { ); } - const translator = new TurnTranslator({ dynamicToolNames: dynamicToolNames(options.tools) }); let afterId: number | undefined; if (action.kind === "new-turn") { @@ -205,30 +284,87 @@ export class CoderLanguageModel implements LanguageModelV3 { // reading until the client tool calls have actually been emitted (bounded // by a safety counter, since the stream is a live subscription). let sinceRequiresAction = 0; - for await (const ev of this.#config.client.streamEvents(chatId, { afterId, signal })) { - for (const part of translator.ingest(ev)) yield part; - const status = translator.terminalStatus; - if (status) { - if (status !== "requires_action") break; - if (translator.clientToolCallSeen) break; - if (++sinceRequiresAction > 200) break; + try { + for await (const ev of this.#config.client.streamEvents(chatId, { afterId, signal })) { + for (const part of translator.ingest(ev)) yield part; + const status = translator.terminalStatus; + if (status) { + if (status !== "requires_action") break; + if (translator.clientToolCallSeen) break; + if (++sinceRequiresAction > 200) break; + } } + } catch (err) { + // Abort surfaces here only if the reader threw instead of closing cleanly; + // prefer the abort/timeout classification. + throwIfAborted(); + // The reader only throws transport-level CoderAgentErrors (socket error / + // unparseable frame). Surface them as a retryable stream failure so a + // caller's `CoderChatError && retryable` retry path catches a dropped + // connection instead of seeing a bare, non-retryable error. + if ( + err instanceof CoderAgentError && + !(err instanceof CoderApiError) && + !(err instanceof CoderChatError) + ) { + throw new CoderChatError({ + message: `Coder chat stream failed mid-turn: ${err.message}`, + kind: "stream_closed", + retryable: true, + }); + } + throw err; } - if (signal?.aborted && !translator.terminalStatus) { - throw signal.reason ?? new DOMException("The operation was aborted.", "AbortError"); + // The stream loop exits cleanly when the socket closes on abort, so classify + // an abort/timeout here before treating the end as a normal/closed turn. + throwIfAborted(); + + // No terminal status: the stream ended before the turn settled. If an + // `error` event arrived (without a trailing `status: error`), fall through to + // finish() so the real error surfaces (unified:"error" + the error part), + // consistent with the `status: error` path; otherwise it's a genuine + // premature close — surface it rather than a clean (truncated) `stop`. + if (!translator.terminalStatus && !translator.error) { + throw new CoderChatError({ + message: + "Coder chat stream ended before the turn settled (connection closed or the server ended the stream without a terminal status).", + kind: "stream_closed", + retryable: true, + }); } for (const part of translator.finish()) yield part; + } catch (err) { + // A timeout/caller abort during the REST phase (createChat / message / + // tool-results / model resolution) rejects the fetch before the stream loop; + // reclassify so the documented retryable timeout/abort contract still holds. + throwIfAborted(); + throw err; + } finally { + // Advance the cursor on every exit (success, abort, error) so resuming the + // same chat doesn't re-read messages already streamed this turn. if (translator.maxMessageId > this.#lastSeenMessageId) this.#lastSeenMessageId = translator.maxMessageId; - } finally { + signal?.removeEventListener("abort", interrupt); + // Teardown of an unsettled turn — stream cancel(), premature close, or an + // abort the listener didn't cover — interrupt the server so it stops. + if (!translator.terminalStatus) interrupt(); this.#inFlight = false; } } async doStream(options: LanguageModelV3CallOptions): Promise { - const gen = this.#runTurn(options); + // A consumer can tear the stream down via ReadableStream.cancel() without + // aborting options.abortSignal. Route cancel through an abort so the turn + // interrupts the server run and the blocked stream reader unblocks — a bare + // gen.return() would deadlock on a pending socket read and never reach the + // interrupt, leaking the workspace. + const cancelController = new AbortController(); + const abortSignal = options.abortSignal + ? AbortSignal.any([options.abortSignal, cancelController.signal]) + : cancelController.signal; + const gen = this.#runTurn({ ...options, abortSignal }); const stream = new ReadableStream({ async pull(controller) { try { @@ -236,11 +372,17 @@ export class CoderLanguageModel implements LanguageModelV3 { if (done) controller.close(); else controller.enqueue(value); } catch (err) { - controller.error(err); + // A consumer-initiated cancel aborts the turn (to interrupt the server + // and unblock the reader); that surfaces here as the turn's AbortError, + // but it's an intentional teardown, so end the stream cleanly rather + // than erroring it. A caller's own abortSignal still errors as usual. + if (cancelController.signal.aborted) controller.close(); + else controller.error(err); } }, async cancel() { - await gen.return(); + cancelController.abort(); + await gen.return().catch(() => {}); }, }); return { stream }; diff --git a/packages/agent/test/unit/agent.test.ts b/packages/agent/test/unit/agent.test.ts index 4958b1b..154e288 100644 --- a/packages/agent/test/unit/agent.test.ts +++ b/packages/agent/test/unit/agent.test.ts @@ -7,6 +7,7 @@ import { type UploadedChatFile, } from "../../src/coder/client.js"; import { CoderAgent } from "../../src/agent/coder-agent.js"; +import { CoderAgentError, CoderChatError } from "../../src/errors.js"; import { CoderLanguageModel } from "../../src/model/language-model.js"; import type { Chat, @@ -36,16 +37,7 @@ class FakeClient { async createChat(req: CreateChatRequest): Promise { this.createdChats.push(req); - return { - id: "chat-1", - organization_id: req.organization_id, - owner_id: "u", - title: "t", - status: "running", - created_at: "", - updated_at: "", - archived: false, - }; + return chatStub("chat-1", req.organization_id); } async createChatMessage(): Promise { @@ -105,6 +97,59 @@ function status(s: string): ChatStreamEvent { return { type: "status", chat_id: "chat-1", status: { status: s as never } }; } +function chatStub(id: string, organizationId = "org-1"): Chat { + return { + id, + organization_id: organizationId, + owner_id: "u", + title: "t", + status: "running", + created_at: "", + updated_at: "", + archived: false, + }; +} + +/** Resolves once the signal aborts (mirrors how the real WS reader unblocks). */ +function waitForAbort(signal: AbortSignal | undefined): Promise { + return new Promise((resolve) => { + if (signal?.aborted) return resolve(); + signal?.addEventListener("abort", () => resolve(), { once: true }); + }); +} + +/** Read a stream to completion (or until it errors). */ +async function drain(reader: ReadableStreamDefaultReader): Promise { + while (!(await reader.read()).done) { + /* discard */ + } +} + +/** + * A fake client whose single turn yields one non-terminal event then never + * settles until its signal aborts — for exercising cancellation/timeout. The + * returned `interrupted` array records every `interruptChat` call. + */ +function stallingClient(onRunning?: () => void): { client: unknown; interrupted: string[] } { + const interrupted: string[] = []; + const client = { + resolveModelConfigId: async () => undefined, + createChat: async () => chatStub("chat-1"), + interruptChat: async (id: string) => { + interrupted.push(id); + return chatStub(id); + }, + archiveChat: async () => {}, + streamEvents: (_id: string, opts?: { signal?: AbortSignal }) => + (async function* () { + yield status("running"); + onRunning?.(); + await waitForAbort(opts?.signal); + })(), + }; + return { client, interrupted }; +} + function makeAgent>(fake: FakeClient, tools?: T) { return new CoderAgent({ client: fake as unknown as CoderChatClient, @@ -331,7 +376,186 @@ describe("CoderAgent file uploads", () => { }); }); +describe("CoderAgent cancellation & failures", () => { + it("interrupts the server run when the caller aborts mid-turn", async () => { + let reachedStream!: () => void; + const midStream = new Promise((r) => { + reachedStream = r; + }); + const { client, interrupted } = stallingClient(reachedStream); + const agent = new CoderAgent({ + client: client as CoderChatClient, + organizationId: "org-1", + }); + + const ac = new AbortController(); + const p = agent.generate({ prompt: "hi", abortSignal: ac.signal }); + await midStream; + ac.abort(); + + await expect(p).rejects.toThrow(); + // Aborting must stop the *server* run, not merely close the socket. + expect(interrupted).toEqual(["chat-1"]); + }); + + it("interrupts and errors a turn that exceeds requestTimeoutMs", async () => { + const { client, interrupted } = stallingClient(); + const model = new CoderLanguageModel({ + client: client as CoderChatClient, + organizationId: "o", + requestTimeoutMs: 30, + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + } as never); + await expect(drain(stream.getReader())).rejects.toMatchObject({ + name: "CoderChatError", + kind: "timeout", + }); + expect(interrupted).toEqual(["chat-1"]); + }); + + it("errors (not a silent stop) when the stream ends before a terminal status", async () => { + // No terminal status — the socket closed mid-run. + const fake = new FakeClient([[status("running"), textPart("partial…")]]); + const model = new CoderLanguageModel({ + client: fake as unknown as CoderChatClient, + organizationId: "o", + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + } as never); + await expect(drain(stream.getReader())).rejects.toMatchObject({ + name: "CoderChatError", + kind: "stream_closed", + }); + }); + + it("re-throws a caller's own AbortSignal.timeout as an abort, not a coder timeout", async () => { + // Caller supplies their own deadline; the agent has no requestTimeoutMs. The + // abort must surface as the caller's TimeoutError, not be rewritten into a + // bogus CoderChatError(kind:"timeout", "…undefined ms…"). + const { client, interrupted } = stallingClient(); + const model = new CoderLanguageModel({ + client: client as CoderChatClient, + organizationId: "o", + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + abortSignal: AbortSignal.timeout(30), + } as never); + const err = await drain(stream.getReader()).then( + () => undefined, + (e) => e as { name?: string }, + ); + expect(err?.name).toBe("TimeoutError"); + expect(err).not.toBeInstanceOf(CoderChatError); + expect(interrupted).toEqual(["chat-1"]); + }); + + it("surfaces a mid-turn stream transport error as a retryable stream_closed", async () => { + const client = { + resolveModelConfigId: async () => undefined, + createChat: async () => chatStub("chat-1"), + interruptChat: async (id: string) => chatStub(id), + streamEvents: () => + (async function* () { + yield status("running"); + throw new CoderAgentError("chat stream socket error"); + })(), + }; + const model = new CoderLanguageModel({ + client: client as unknown as CoderChatClient, + organizationId: "o", + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + } as never); + await expect(drain(stream.getReader())).rejects.toMatchObject({ + name: "CoderChatError", + kind: "stream_closed", + retryable: true, + }); + }); + + it("classifies a timeout during chat creation as a retryable timeout error", async () => { + const client = { + resolveModelConfigId: async () => undefined, + // Hangs until the per-turn timeout aborts the signal, then rejects like fetch. + createChat: async (_req: unknown, signal?: AbortSignal) => { + await waitForAbort(signal); + throw signal?.reason ?? new Error("aborted"); + }, + interruptChat: async (id: string) => chatStub(id), + streamEvents: () => (async function* () {})(), + }; + const model = new CoderLanguageModel({ + client: client as unknown as CoderChatClient, + organizationId: "o", + requestTimeoutMs: 30, + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + } as never); + await expect(drain(stream.getReader())).rejects.toMatchObject({ + name: "CoderChatError", + kind: "timeout", + }); + }); + + it("interrupts the server run when the stream is cancelled mid-turn", async () => { + let reached!: () => void; + const midStream = new Promise((r) => { + reached = r; + }); + const { client, interrupted } = stallingClient(reached); + const model = new CoderLanguageModel({ + client: client as CoderChatClient, + organizationId: "o", + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + } as never); + const reader = stream.getReader(); + await reader.read(); // stream-start + const pending = reader.read(); // drive to mid-stream, then block on the reader + await midStream; + await reader.cancel(); // teardown without aborting a caller signal + await pending.catch(() => {}); + + expect(interrupted).toEqual(["chat-1"]); + }); +}); + describe("CoderLanguageModel guards", () => { + it("warns that responseFormat is not enforced server-side", async () => { + const fake = new FakeClient([ + [status("running"), msg(2, "assistant", [{ type: "text", text: "{}" }]), status("waiting")], + ]); + const model = new CoderLanguageModel({ + client: fake as unknown as CoderChatClient, + organizationId: "o", + }); + + const { stream } = await model.doStream({ + prompt: [{ role: "user", content: [{ type: "text", text: "hi" }] }], + responseFormat: { type: "json" }, + } as never); + const reader = stream.getReader(); + const first = await reader.read(); + expect(first.value).toMatchObject({ type: "stream-start" }); + expect((first.value as { warnings: unknown[] }).warnings).toContainEqual( + expect.objectContaining({ type: "unsupported", feature: "responseFormat" }), + ); + await drain(reader); + }); + it("throws on a prompt with no user message or tool results", async () => { const model = new CoderLanguageModel({ client: new FakeClient([]) as unknown as CoderChatClient, @@ -352,16 +576,9 @@ describe("CoderLanguageModel guards", () => { }); const blocking = { resolveModelConfigId: async () => undefined, - createChat: async () => ({ - id: "c1", - organization_id: "o", - owner_id: "u", - title: "t", - status: "running", - created_at: "", - updated_at: "", - archived: false, - }), + createChat: async () => chatStub("c1", "o"), + interruptChat: async (id: string) => chatStub(id), + archiveChat: async () => {}, // Yields one non-terminal event then blocks (on a releasable gate), // keeping turn 1 in-flight while we attempt a concurrent turn 2. streamEvents: async function* () { diff --git a/packages/provider/README.md b/packages/provider/README.md index 1cbb378..ee2cc44 100644 --- a/packages/provider/README.md +++ b/packages/provider/README.md @@ -35,6 +35,16 @@ audits usage per user. This package lets the Vercel AI SDK speak to it natively, your developers never handle raw provider keys — they authenticate with their Coder token and the deployment decides which models and providers are available. +## Provider vs. Agent + +This package is for **plain model calls** — `generateText`, `streamText`, and +`generateObject` (schema‑constrained structured output). If you need Coder's +**server‑side agent** — the multi‑step tool loop, built‑in tools, MCP servers, or +workspace file/shell tools — use **[`@coder/ai-sdk-agent`](../agent)** instead. +Rule of thumb: **need a model → provider; need server‑side tools, MCP, or a +workspace → Agent.** They compose: use the provider for pure text/JSON steps and +the Agent for the tool‑driven ones. + ## Install ```bash