AI-Critic

A modular video generation pipeline that creates art criticism dialogues and voice narration using Claude and ElevenLabs, designed to work with Unreal Engine's Audio Driven Animation for MetaHumans.

Features

• 🎨 AI Art Analysis: Generate sophisticated art criticism dialogue using Claude
• 🎙️ Voice Generation: Convert dialogue to high-quality MP3 speech using ElevenLabs
• 🎭 Unreal Integration: Designed to work seamlessly with Unreal Engine's Audio Driven Animation for MetaHumans
• 📁 Workspace Management: Organized file structure for incremental video production
• 🔧 Modular Pipeline: Run individual steps or the complete workflow
• 💰 Cost Efficient: Re-run individual steps without burning API credits

Setup

Prerequisites

1. Node.js Environment

   npm install

2. Unreal Engine (for MetaHuman animation using Audio Driven Animation)

   • Download from Epic Games Launcher
   • Import MetaHuman characters from MetaHuman Creator
   • Enable Audio Driven Animation plugin

3. API Keys

   # Required for script generationexport ANTHROPIC_API_KEY='your-anthropic-api-key'# Required for voice generationexport ELEVENLABS_API_KEY='your-elevenlabs-api-key'

4. Python Environment (for Unreal Engine automation)

   # One-time setup
   ./setup_python.sh
   # For each sessionsource venv/bin/activate

Project Structure

src/
├── js/ # JavaScript pipeline (Text → Voice)
│ ├── generateFullPipeline.js # Main pipeline orchestrator (script + voice)
│ ├── generateScriptText.js # Claude API integration (text generation only)
│ ├── voiceGeneration.js # ElevenLabs API integration (speech synthesis)
│ ├── parseScript.js # Script parsing utilities
│ └── workspaceUtils.js # Workspace management utilities
└── python/ # Python scripts (Unreal automation)
└── requirements.txt # Python dependencies
workspace/ # Auto-generated workspaces (git-ignored)
└── artwork_name_timestamp/
├── script.txt # Raw generated script
├── dialogue.json # Parsed dialogue lines
└── audio/ # Generated voice files (MP3 format)
├── 01_elena.mp3 # Elena's first line
├── 02_marcus.mp3 # Marcus's first line
└── ... # Alternating dialogue

Usage

Full Pipeline (Recommended)

Generate both script and voice files in one command:

node src/js/generateFullPipeline.js "sample/sample_image.png""Electric Dreams""AI"

Output:

• Creates timestamped workspace folder
• Generates script with Claude API
• Parses dialogue into structured format
• Creates voice files with ElevenLabs API
• Organizes all files for video production

Individual Steps

1. Script Generation Only

Generate script and dialogue files without voice synthesis:

node src/js/generateScriptText.js "sample/sample_image.png""Electric Dreams""AI"

Use when:

• Testing different prompts/artwork
• Iterating on script content
• Saving ElevenLabs API costs

2. Voice Generation Only

Generate voice files from existing dialogue:

node src/js/voiceGeneration.js "workspace/electric_dreams_2025-08-09T00-42-24"

Use when:

• Re-generating with different voices
• Fixing audio quality issues
• Testing different voice settings
• Saving Claude API costs

Example Workflow

# 1. Generate script first (uses Claude API)
node src/js/generateScriptText.js "my-artwork.jpg""Starry Night""Van Gogh"# 2. Iterate on voice generation (uses ElevenLabs API)
node src/js/voiceGeneration.js "workspace/starry_night_2025-08-09T12-00-00"# 3. Re-run voice with adjustments if needed
node src/js/voiceGeneration.js "workspace/starry_night_2025-08-09T12-00-00"

Audio Quality

The voice generation system creates high-quality MP3 files optimized for Unreal Engine's Audio Driven Animation:

High-Quality MP3 Files

• Generated directly from ElevenLabs API in high-quality format
• Professional audio quality suitable for video production
• Compatible with Unreal Engine's Audio Driven Animation system
• Optimized for MetaHuman facial animation processing

Benefits

• Cost Efficient: Single API call per dialogue line
• Quality Preservation: Direct high-quality output from ElevenLabs
• Unreal Integration: Format compatible with Audio Driven Animation
• Simplified Workflow: No additional transcoding required

Voice Characters

The system uses two distinct art critics:

• Elena (CRITIC A): Academic, formal voice - focuses on art history and technique
• Marcus (CRITIC B): Contemporary, conversational voice - focuses on cultural relevance

File Outputs

Each generation creates a workspace with:

workspace/artwork_name_timestamp/
├── script.txt # Full dialogue script
├── dialogue.json # Structured dialogue data
└── audio/ # Generated voice files (MP3 format)
├── 01_elena.mp3 # Elena's first line
├── 02_marcus.mp3 # Marcus's first line
└── ... # Alternating dialogue

Error Handling

• Missing API Keys: Scripts check for required environment variables
• Invalid Workspaces: Voice generation validates workspace structure
• Partial Failures: Individual voice line failures don't stop the process
• File Conflicts: Timestamped folders prevent overwrites

Cost Optimization

• Script-only runs: Test content without voice generation costs
• Voice-only runs: Iterate on audio without re-generating scripts
• Workspace reuse: Point voice generation at any existing folder
• Batch processing: Generate multiple artworks efficiently
• Single format output: Streamlined workflow reduces complexity and costs

Unreal Engine Audio Driven Animation

The generated MP3 files are designed to work with Unreal Engine's new Audio Driven Animation feature for MetaHumans:

Setup in Unreal Engine

1. Enable Audio Driven Animation Plugin

   • In Unreal Engine, go to Edit → Plugins
   • Search for "Audio Driven Animation"
   • Enable the plugin and restart the engine

2. Import MetaHuman Character

   • Use MetaHuman Creator to design your characters
   • Import the characters into your Unreal project
   • Ensure the MetaHuman has proper facial rig setup

3. Configure Audio Driven Animation

   • Add Audio Driven Animation component to your MetaHuman
   • Configure the audio analysis settings
   • Map audio frequencies to facial blendshapes

Workflow

# 1. Generate script and voice files
node src/js/generateFullPipeline.js "artwork.jpg""Title""Artist"# 2. Import MP3 files into Unreal Engine# 3. Set up Audio Driven Animation on MetaHuman characters# 4. Configure audio analysis and blendshape mapping# 5. Use Sequencer to create your animated scenes# 6. Render with Movie Render Queue

Next Steps

Generated files are ready for:

• Unreal Engine: Import MP3 files and configure Audio Driven Animation
• MetaHuman Setup: Apply audio-driven facial animation to your characters
• Video Production: Use Sequencer and Movie Render Queue for automated rendering
• Distribution/Publishing: Create professional-quality art criticism videos