Skip to content
Closed
4 changes: 4 additions & 0 deletions .agentsroom/.gitignore
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
# AgentsRoom: personal files (not committed to git)
*-personal.json
agents-local.json
sessions/
10 changes: 10 additions & 0 deletions .agentsroom/agents.json
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
[
{
"role": "fullstack",
"model": "opus",
"customName": "Full-Stack Developer",
"isPersonal": false,
"id": "agent-1776361243376-3sekdc",
"claudeSessionId": "96773a93-be2a-45a9-a732-ceb224d3d0e5"
}
]
4 changes: 4 additions & 0 deletions .agentsroom/prompts.json
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
{
"folders": [],
"prompts": []
}
21 changes: 21 additions & 0 deletions .changeset/media-edit-from.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
---
'@tanstack/ai': minor
'@tanstack/ai-openai': minor
'@tanstack/ai-gemini': minor
'@tanstack/ai-grok': minor
'@tanstack/ai-fal': minor
'@tanstack/ai-client': minor
---

feat: first-class follow-up edits for generated media.

`generateVideo({ ..., previousJobId })` edits a previously generated video instead of generating from scratch. Callers always pass the prior generation's job id; adapters decide how to consume it via a `VideoAdapter` edit-kind map:

- `'job'` — reference the id server-side (OpenAI Sora 2 / Sora 2 Pro remix; Gemini Omni Flash, which maps `previousJobId` onto the Interactions API's `previous_interaction_id` wire field — that field is omitted from Omni `modelOptions`)
- `'media'` — resolve the finished clip via `getVideoUrl(previousJobId)` (xAI `grok-imagine-video` → `/videos/edits`; fal video-to-video endpoints such as `xai/grok-imagine-video/edit-video` and Seedance 2.0 reference-to-video). Fal generate endpoints with a known edit sibling (e.g. Grok text/image-to-video) resolve on the generate model, then submit to the edit endpoint.

Non-editing models (Veo, `grok-imagine-video-1.5`) reject `previousJobId` at compile time. Sora remix and Grok edits accept only a prompt — `size` / `duration` / media inputs are rejected because the output inherits them from the source video.

`generateImage({ ..., previousImage })` is the image-side counterpart: pass a prior result's `GeneratedImage` (or an array, or the whole result) and it is prepended to the prompt as an image part, flowing through each adapter's existing edit path; type-gated to models that accept image inputs.

Breaking for hand-rolled (non-`BaseVideoAdapter`) `VideoAdapter` implementations: the interface gains `supportedEditKind(): 'job' | 'media' | undefined` (and a 7th, defaulted `TModelEditByName` generic — existing 6-argument instantiations keep compiling). `BaseVideoAdapter` supplies a default returning `undefined`, plus `resolvePreviousJobUrl(previousJobId)`. New exports include `VideoEditKind`, `ModelEditKindByName`, `VideoPreviousJobIdForAdapter`, `ImagePreviousSource`, `ImagePreviousImageForModel`, `generatedImageToImagePart`, `generatedVideoUrlToVideoPart`. Client wire types: `VideoGenerateInput.previousJobId`, `ImageGenerateInput.previousImage`.
4 changes: 2 additions & 2 deletions docs/config.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -480,13 +480,13 @@
"label": "Image Generation",
"to": "media/image-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-19"
"updatedAt": "2026-08-21"
},
{
"label": "Video Generation",
"to": "media/video-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-20"
"updatedAt": "2026-08-21"
},
{
"label": "Generation Hooks",
Expand Down
78 changes: 71 additions & 7 deletions docs/media/image-generation.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -99,13 +99,14 @@ Image URLs expire after 24 hours; pass `response_format: 'b64_json'` in `modelOp

All image adapters support these common options:

| Option | Type | Description |
| ---------------- | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `adapter` | `ImageAdapter` | Image adapter instance with model (required) |
| `prompt` | `string \| MediaPromptPart[]` | Description of the image to generate (required). A plain string, or — on models that support image-conditioned generation — an ordered array of content parts interleaving text with image inputs. See [Image-Conditioned Generation](#image-conditioned-generation) below. |
| `numberOfImages` | `number` | Number of images to generate |
| `size` | `string` | Size of the generated image in WIDTHxHEIGHT format |
| `modelOptions?` | `object` | Model-specific options (renamed from `providerOptions`) |
| Option | Type | Description |
| ----------------- | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `adapter` | `ImageAdapter` | Image adapter instance with model (required) |
| `prompt` | `string \| MediaPromptPart[]` | Description of the image to generate (required). A plain string, or — on models that support image-conditioned generation — an ordered array of content parts interleaving text with image inputs. See [Image-Conditioned Generation](#image-conditioned-generation) below. |
| `numberOfImages` | `number` | Number of images to generate |
| `size` | `string` | Size of the generated image in WIDTHxHEIGHT format |
| `previousImage?` | `GeneratedImage \| GeneratedImage[] \| { images }` | A previously generated image (or images) to edit — prepended to the prompt as image input(s). Only offered (at compile time) for models that accept image inputs — see [Editing generated images](#editing-generated-images-previousimage). |
| `modelOptions?` | `object` | Model-specific options (renamed from `providerOptions`) |

### Size Options

Expand DownExpand Up@@ -217,6 +218,69 @@ The accepted part types are narrowed **per model at compile time**: passing
an image part to a text-only model (e.g. `dall-e-3`, Imagen) is a type
error, not just a runtime throw.

### Editing generated images (previousImage)

To run a **follow-up edit** on something you just generated, pass the prior
result's image as `previousImage` — sugar that prepends it to the prompt as an
image part, so it flows through the model's regular edit path (OpenAI
`/images/edits`, Gemini `generateContent`, xAI `/images/edits`, fal).

**Server:**

```typescript
import { generateImage } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

const adapter = openaiImage('gpt-image-2')

const first = await generateImage({ adapter, prompt: 'A city street at dusk' })

const edited = await generateImage({
adapter,
prompt: 'Same scene, but make it rain',
previousImage: first.images[0],
})
```

`previousImage` accepts a single `GeneratedImage`, an array of them, or the
whole prior result (`{ images }`). URL results pass through as `url`
sources (`data:` URLs are decomposed into raw bytes for adapters that
upload files); `b64Json` results become `data` sources with the mime type
sniffed from the payload (defaulting to `image/png`). Like image parts, the
option is offered **per model at compile time** — text-only models
(`dall-e-3`, Imagen) reject it as a type error.

**Client** — the hook's `ImageGenerateInput.previousImage` is a wire-friendly
`{ url? }` / `{ b64Json? }` shape; your server route should narrow it back
to a `GeneratedImage` before calling `generateImage`:

```tsx
import { useGenerateImage, fetchServerSentEvents } from '@tanstack/ai-react'

function ImageEditor() {
const { generate, result, isLoading } = useGenerateImage({
connection: fetchServerSentEvents('/api/generate/image'),
})

const handleEdit = () => {
const image = result?.images[0]
if (!image) return
void generate({
prompt: 'Same scene, but make it rain',
previousImage: image.url
? { url: image.url }
: { b64Json: image.b64Json },
})
Comment on lines +253 to +273

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Include the server endpoint half of this example.

The client snippet sends a wire-friendly { url? } / { b64Json? } value and explicitly says the server must narrow it, but no route showing that conversion is provided. Add a minimal server endpoint that validates the payload and passes a GeneratedImage to generateImage.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/media/image-generation.md` around lines 219 - 239, Extend the
image-generation example with a minimal server endpoint matching the client
request, validating the wire-friendly previousImage shape, narrowing it to a
GeneratedImage, and passing it to generateImage. Reference the route handler and
generateImage usage, and show handling for invalid payloads and both URL and
base64 image variants.

Source: Coding guidelines

}

return (
<button onClick={handleEdit} disabled={isLoading || !result}>
Edit
</button>
)
}
```

### Referencing images from your prompt

**Your prompt text is always sent verbatim — the SDK never injects or
Expand Down
Loading
Loading