AI Video Studio

OpenClaw skill for AI video generation + editing. Generate video clips with Seedance 2.0, edit with CapCut Mate or FFmpeg, and export — all server-side, no desktop app needed.

Features

• AI Video Generation — Text/image-to-video via ByteDance Seedance 2.0 (Doubao)
• Smart Editing — Auto-compose multi-shot storyboards with captions, BGM, and transitions
• Multi Renderer — FFmpeg (default MP4), CapCut Mate, local CapCut/Jianying draft manifests
• Draft Backends — Storyboards with real-shot assets can become CapCut/Jianying draft plans for later pyCapCut / pyJianYingDraft execution
• Social Reply Queue — Normalize comments from Douyin/TikTok/Instagram/YouTube exports into a deduped SQLite review queue with safe draft replies
• One-command Drama / Product Promo — drama subcommand runs a four-layer pipeline (script → characters/scenes → storyboard → render) with autonomous narrative-template selection; methodology ported from huobao-drama (CC BY-NC-SA 4.0, see skills/ai-video-studio/reference/huobao/ATTRIBUTION.md)
• OpenClaw Native — Install as a skill, invoke via natural language through any channel (Telegram, WhatsApp, CLI)

One-command Scenario Video

# Scenario-based product promotion
python3 scripts/studio.py drama \
--mode product \
--idea "Smart thermos for commuters" \
--scenario "morning run / subway / office" \
--highlights "12h heat retention, aerospace steel" \
--shots 6 --duration 5 --ratio 9:16 --lang en --run
# Short drama
python3 scripts/studio.py drama \
--mode shortdrama \
--idea "Café reunion: one sentence uncovers a three-year misunderstanding" \
--shots 6 --duration 8 --ratio 9:16 --lang zh --run

Requires ANTHROPIC_API_KEY (Sonnet executor + Opus advisor) in addition to SEEDANCE_API_KEY.

Quick Start

As an OpenClaw Skill (recommended)

# Copy to your OpenClaw workspace
cp -r skills/ai-video-studio ~/.openclaw/workspace/skills/
# Set your Seedance API keyexport SEEDANCE_API_KEY="your-key-here"# Talk to your OpenClaw agent:# "Generate a 3-shot product video, 16:9, 5 seconds each, with captions"

Standalone CLI

# Generate a single clip
python3 scripts/studio.py generate \
--prompt "A cat yawning in golden sunlight" \
--ratio 16:9 --duration 5
# Render existing clips into final video (FFmpeg)
python3 scripts/studio.py render \
--videos clip1.mp4,clip2.mp4 \
--captions "Scene 1,Scene 2" \
--bgm https://example.com/music.mp3 \
--ratio 16:9
# Full pipeline from storyboard
python3 scripts/studio.py pipeline storyboard.json
# Create local CapCut/Jianying draft manifests from storyboard assets
python3 scripts/studio.py draft storyboard.json --backend capcut --draft-root "/path/to/CapCut Drafts"
python3 scripts/studio.py draft storyboard.json --backend jianying --draft-root "/path/to/JianyingPro Drafts"# Create 3 variants × zh/en local draft manifests from a product brief and real-shot assets
python3 scripts/studio.py template-draft brief.json \
--assets "/path/to/demo1.mp4,/path/to/demo2.mp4" \
--backend capcut --draft-root "/path/to/CapCut Drafts" \
--template ugc_hook_cta --locales zh,en \
--platforms tiktok,reels,shorts --creative-copy local_rules --variants 3
# Turn social comment exports into reviewable reply jobs
python3 scripts/social_replies.py comments.json --db social.sqlite --brand ClipForge --url https://example.com/buy

Storyboard Format

{
"title": "Product Ad",
"ratio": "16:9",
"renderer": "ffmpeg",
"shots": [
{
"prompt": "Modern smartphone floating in space with golden particles",
"duration": 5,
"caption": "Next-Gen Design"
},
{
"prompt": "Hands holding smartphone with holographic UI elements",
"duration": 5,
"caption": "Intelligence at Your Fingertips"
}
],
"bgm_url": null
}

For real-shot material entering the local draft backend, put a local asset path on each shot:

{
"title": "UGC Product Demo",
"ratio": "9:16",
"renderer": "capcut_draft",
"draft_root": "/path/to/CapCut Drafts",
"lang": "zh",
"shots": [
{
"asset": "/absolute/path/to/real-shoot-01.mp4",
"duration": 5,
"caption": "三秒看懂",
"voiceover": "/absolute/path/to/voice.zh.mp3"
}
],
"locales": {
"en": {
"shots": [{"caption": "Get it in 3 sec"}]
}
}
}

This writes one draft-plan JSON per locale. The next execution layer can map those operations to pycapcut or pyJianYingDraft.

Multilingual Template Drafts

scripts/draft_templates.py turns a product brief plus real-shot assets into reusable CapCut/Jianying draft packages. The first built-in template is ugc_hook_cta:

• 3 slots: hook, benefit, CTA
• 9:16 default canvas and lower-third-safe captions
• variants controls how many creative cuts are produced
• locales controls language manifests, for example zh,en
• platforms controls social platform packaging, for example tiktok,reels,shorts
• each platform profile carries safe-zone metadata for captions and UI overlays
• --creative-copy local_rules generates platform/language/variant-specific hook, benefit, and CTA copy without any LLM API
• --creative-copy local_llm|anthropic|doubao runs a JSON-in/JSON-out command adapter configured by CREATIVE_COPY_*_COMMAND
• no Seedance, no Anthropic, no TTS required; it uses existing local assets

Example brief:

{
"product_name": "智能保温杯",
"brand_name": "ClipForge",
"audience": "通勤族",
"selling_points": ["12小时保温", "一键开盖"],
"cta": "现在领取试用",
"slug": "mug-campaign"
}

The HTTP service exposes the same local-first template flow at POST /draft/template:

{
"brief": {
"product_name": "智能保温杯",
"brand_name": "ClipForge",
"audience": "通勤族",
"selling_points": ["12小时保温"],
"cta": "现在领取试用",
"slug": "mug-campaign"
},
"assets": ["/path/to/demo.mp4"],
"backend": "capcut",
"template_id": "ugc_hook_cta",
"locales": ["zh", "en"],
"platforms": ["tiktok", "reels", "shorts"],
"creative_copy_mode": "local_rules",
"variants": 3
}

Creative copy adapters:

Adapter	Behavior
local / local_rules	Built-in no-key rules
local_llm	Runs CREATIVE_COPY_LOCAL_LLM_COMMAND
anthropic	Runs CREATIVE_COPY_ANTHROPIC_COMMAND
doubao	Runs CREATIVE_COPY_DOUBAO_COMMAND

Command adapters receive JSON on stdin: brief, platforms, locales, variants. They must print a JSON list shaped like creative_copy.build_copy_matrix(...).

Social Comment Reply Queue

scripts/social_replies.py is the safe core for the "all social media auto-reply" product line. It does not log in or send replies by itself. It:

• accepts comment exports from Douyin/TikTok/Instagram/YouTube-style JSON fields
• normalizes platform, comment_id, post_id, author, text, and created_at
• dedupes comments in SQLite by (platform, comment_id)
• drafts replies into pending_review
• routes refund, complaint, scam, legal, and similar terms to needs_human

Later platform adapters can plug into this queue as CommentProvider and ReplySender layers, including the TzFilm-style "export comments → plan replies → controlled browser send" pattern.

The HTTP service exposes the same safe planning layer at POST /social/replies/plan:

{
"comments": [
{
"platform": "youtube",
"comment_id": "yt-1",
"video_id": "short-1",
"author": "lee",
"text": "How much is this?"
}
],
"db_path": "social.sqlite",
"brand_name": "ClipForge",
"product_url": "https://example.com/buy"
}

Architecture

User Intent
│
▼
┌─────────────┐ ┌──────────────┐
│ Seedance │───▶│ Video Clips │
│ 2.0 API │ │ (.mp4 × N) │
└─────────────┘ └──────┬───────┘
│
┌──────▼───────┐
│ Renderer │
│ │
│ ┌──────────┐ │
│ │ FFmpeg │ │ ← Default: concat + subtitle burn + BGM mix
│ └──────────┘ │
│ ┌──────────┐ │
│ │ CapCut │ │ ← Optional: Jianying draft with effects
│ │ Mate │ │
│ └──────────┘ │
└──────┬───────┘
│
┌──────▼───────┐
│ Final MP4 │
└──────────────┘

Commands

Command	Description	Dependencies
generate	Generate a single clip via Seedance 2.0	Seedance API key
compose	Create Jianying draft via CapCut Mate	capcut-mate service
render	FFmpeg direct render (concat + subs + BGM)	ffmpeg
pipeline	End-to-end from storyboard JSON	Seedance + ffmpeg
template-draft	Product brief + local assets → multilingual draft manifests	stdlib + draft backend
social_replies.py	Turn comment exports into reviewable reply jobs	stdlib SQLite

Environment Variables

Variable	Purpose	Default
SEEDANCE_API_KEY	Doubao Seedance API key	Built-in fallback
CAPCUT_MATE_URL	CapCut Mate service URL	http://127.0.0.1:30000
CAPCUT_API_KEY	Jianying cloud render key (optional)	—
CAPCUT_DRAFT_ROOT	Local CapCut draft folder for capcut_draft	—
JIANYING_DRAFT_ROOT	Local Jianying draft folder for jianying_draft	—
MEDIA_DIR	Output directory	/root/.openclaw/media

Prerequisites

• Python 3.11+
• ffmpeg (for render mode)
• requests + pyyaml (pip)
• CapCut Mate (optional, for compose mode)
• pyCapCut and pyJianYingDraft are the target local draft execution libraries. Current integration writes normalized draft manifests first, then maps to these libraries in the executor layer.
• Seedance 2.0 API access (for generate/pipeline)

FFmpeg Render Capabilities

• Multi-clip concatenation with automatic resolution normalization
• SRT subtitle burn-in (white text, black outline, bottom-center)
• BGM mixing (original audio 100% + BGM 30%)
• H.264 + AAC encoding, fast preset
• Sub-second render time for short videos

License

MIT — see LICENSE.

Built With

• OpenClaw — AI agent framework
• Seedance 2.0 — ByteDance video generation
• CapCut Mate — Open-source Jianying automation
• FFmpeg — Video processing