VisualFries is a Svelte 5 library for programmatically creating (not just) bite-sized video content for social media. Built on PIXI.js and GSAP (for powerful animations), it's designed primarily for headless rendering of dynamic scenes.
A key feature of VisualFries is its unique approach to text rendering. It uses SVG <foreignObject> to render standard HTML and CSS, allowing you to style scenes and create complex text animations with familiar tools and the full power of GSAP.
⚠️ Alpha Software Notice⚠️ This library is currently in an alpha stage. It is not recommended for production use. The API is unstable and may change without notice. There are known bugs and likely many unknown ones. I welcome feedback and bug reports, but please be aware of its experimental nature.
VisualFries is currently a solo project maintained by me, its creator. I use this as a backbone of my SaaS, ContentFries.
This means development is driven by a singular vision and you have a direct line to the person who built it. It also means resources are limited. I'm opening this up to the community to share what I've built, get feedback, and see where it goes. I wholeheartedly welcome contributions, bug reports, and ideas.
- Built for Svelte 5:
- Declarative, JSON-based Scenes: Define your entire video as a structured, type-safe JSON object.
- HTML & CSS for Text: Style text and overlays with standard CSS. No need to learn a proprietary canvas styling API.
- Powerful Animation Engine: Leverage the full GSAP ecosystem for complex, timeline-based animations.
- Component-Based Architecture: Build scenes by composing Videos, Images, Text, Shapes, and Subtitles.
- Fluent Scene Composition: Use composer utilities to programmatically build scenes, layers, and components with a clean, chainable API.
- Headless & Server-Side Ready: Designed for automated, server-side video generation.
- MIT Licensed: Truly open-source and free for all uses.
Prerequisite: Your project must be using Svelte 5 or newer.
npm install visualfriesVisualFries includes a small agent-facing CLI for scene JSON workflows that do not require opening the ContentFries UI.
visualfries validate scene.json
visualfries inspect scene.json --json
visualfries qa scene.json --output ./qa
visualfries doctor --json
visualfries catalog --component TEXT --capabilities --json
visualfries validate scene.json --strict-runtime-support
visualfries explain scene.json --component views-badge --frame 12 --json
visualfries parity scene.json --frames 5,12,35 --rois qa/parity-rois.json --output qa/parity --json
visualfries validate-cues ./cues.json --duration 45 --json
visualfries caption-scene \
--video ./input.mp4 \
--transcript ./transcript.srt \
--preset hidden-engine-center \
--output ./scene.json
visualfries preset-cues --duration 45 --preset hidden-engine-dynamic --output ./cues.json
visualfries apply-cues ./scene.json --cues ./cues.json --output ./scene.with-cues.json
visualfries render ./scene.json --output ./out.mp4For static-heavy scenes, QA first and then use the faster duplicate-aware render:
visualfries render ./scene.json --output ./out.mp4 --skip-duplicatesvisualfries init also creates a compose-ready package with scene.json, cues.json, assets/, qa/, and notes.md.
For a one-command agent render:
visualfries compose \
--video ./input.mp4 \
--transcript ./transcript.srt \
--cue-preset hidden-engine-dynamic \
--cues ./cues.json \
--scene-output ./scene.json \
--qa-output ./qa/frames \
--output ./out.mp4The caption-scene command creates a render-ready VisualFries scene with a full-frame video layer, subtitle layer, asset registry, and subtitle timing data from transcript JSON, SRT, or VTT.
The render command opens a controlled browser renderer, captures the scene frame-by-frame, and encodes MP4 with ffmpeg. Use --frames-only when an agent needs QA frames before encoding.
For Node automations, use the agent-only export:
import{createCaptionScene,inspectScene,normalizeTranscript}from'visualfries/agent';Agent helpers also include timed text overlays:
import{addAgentBrollSequence,addAgentTextOverlays,addAgentTransitions}from'visualfries/agent';constsceneWithOverlays=addAgentTextOverlays({
scene,overlays: [{text: 'LOVE THIS 😍',start: 0.4,end: 1.1,style: 'hook-punch'},{text: 'NECK 🤯',start: 1.1,end: 1.7,style: 'shock-word'}]});constsceneWithBroll=addAgentBrollSequence({
scene,cues: [{url: './broll/profile.mp4',start: 2.0,end: 5.0,type: 'VIDEO'},{url: './broll/chart.png',start: 5.0,end: 7.0,type: 'IMAGE'}]});constsceneWithTransitions=addAgentTransitions({
scene,transitions: [{time: 2.0,style: 'dip-to-black'},{time: 5.0,style: 'swipe-left',color: '#04483D'}]});Start with Authoring Best Practices: native TEXT policy, component decision tree, animation runtime matrix, compositing truth, frame-QA, and known limits. Then see Agent Workflow, Agent Patterns, and the 2026 authoring-system audit.
Visible typography defaults to native TEXT. Do not rasterize ordinary badges/metrics into
IMAGE, and do not add a SHAPE solely as a text background when TEXT can own the treatment.
The best way to use VisualFries is within a Svelte component.
<!-- src/routes/+page.svelte -->
<scriptlang="ts">import { onMount, onDestroy } from'svelte';import { createSceneBuilder, typeISceneBuilder, typeScene } from'visualfries';let canvasContainer:HTMLDivElement;let sceneBuilder:ISceneBuilder;// Define your scene using a declarative JSON objectconst myScene:Scene= { id: 'my-first-scene', version: '2.0', settings: { width: 1080, height: 1080, duration: 10, fps: 30, backgroundColor: '#1a1a1a' }, layers: [// ... your layers and components defined here ], assets: [] };onMount(async () => {if (canvasContainer) {sceneBuilder=awaitcreateSceneBuilder(myScene, canvasContainer, { environment: 'client', autoPlay: true }); } });onDestroy(() => {sceneBuilder?.destroy(); });</script>
<divclass="scene-container"bind:this={canvasContainer} />
<style>.scene-container {width: 540px; /* Example scaled size */height: 540px;border: 1pxsolid#333; }</style>While you can write the scene JSON by hand, the library includes a fluent API to make this process simple and type-safe.
import{createSceneComposer,createLayerComposer,createComponentComposer}from'visualfries';// 1. Create a composer for a TEXT componentconstheadline=createComponentComposer('headline-text','TEXT',{startAt: 0,endAt: 5}).setAppearance({x: 50,y: 100,width: 980,height: 250,text: {fontFamily: 'Montserrat',fontSize: 90,fontWeight: '800',color: '#FFFFFF',textAlign: 'center'}}).setText('Hello, VisualFries!').compose();// 2. Create a layer and add the componentconstmainLayer=createLayerComposer('layer-1').addComponent(headline).compose();// 3. Create the sceneconstmyScene=createSceneComposer('my-scene',{width: 1080,height: 1080,duration: 10,fps: 30}).addLayer(mainLayer).compose();VisualFries needs to load font files (.ttf, .woff2, etc.) to render text. The library uses a flexible font provider chain to allow you to load fonts from any source while retaining default support for Google Fonts.
By default, with no configuration, VisualFries will automatically fetch fonts from Google Fonts. This works out of the box for any client-side application.
Instead of replacing the default behavior, you can add your own font providers to the front of the chain. This is the recommended approach for loading local or custom fonts.
A font provider is a simple async function that receives a font family and must return a Promise<ArrayBuffer | null>. If it returns null, the library will try the next provider in the chain.
Example: Adding a provider for local fonts while keeping Google Fonts as a fallback.
Create your custom provider:
// src/lib/localFontProvider.tsimporttype{FontProvider}from'visualfries';exportconstlocalFontProvider: FontProvider=async(fontFamily)=>{// We only handle fonts that start with our special prefixif(!fontFamily.startsWith('local://')){// For all other fonts (e.g., "Roboto"), we do nothing and// let the next provider in the chain (the Google provider) handle it.returnnull;}constfontName=fontFamily.replace('local://','');constfontUrl=`/fonts/${fontName}.ttf`;// Assuming fonts are in /static/fontstry{constresponse=awaitfetch(fontUrl);if(!response.ok){console.error(`Failed to load local font: ${fontUrl}`);returnnull;}returnresponse.arrayBuffer();}catch(error){console.error(`Error fetching local font: ${fontUrl}`,error);returnnull;}};Configure the fontProviders array:
When initializing the SceneBuilder, provide your custom provider and the default Google Fonts provider. The library will check yours first.
// In your Svelte componentimport{createSceneBuilder,createGoogleFontsProvider}from'visualfries';import{localFontProvider}from'$lib/localFontProvider';constbuilder=awaitcreateSceneBuilder(sceneData,container,{environment: 'client',fontProviders: [// 1. Your provider is checked first.localFontProvider,// 2. If your provider returns null, the default provider is checked next.createGoogleFontsProvider()]});Server mode remains canvas-first by default for backward compatibility:
awaitcreateSceneBuilder(sceneData,container,{environment: 'server'// serverRendererMode defaults to "canvas"});To opt into GPU rendering on server/headless runtimes:
awaitcreateSceneBuilder(sceneData,container,{environment: 'server',serverRendererMode: 'webgl',preferWebGL2: true,powerPreference: 'high-performance'});If WebGL is unavailable or initialization fails, Visualfries automatically falls back to canvas so deterministic render jobs keep running. When deterministic diagnostics are enabled, renderer selection and fallback reason are included in getDiagnosticsReport().
In serverRendererMode: 'webgl', fillBackgroundBlur uses native PIXI.BlurFilter (same rendering path as client mode).
As an early-stage, solo-developed project, contributions are highly encouraged! The best way to contribute right now is by:
- Opening an issue to report a bug or suggest a feature
- Improving the documentation by clarifying confusing sections or adding new examples
- Sharing your creations! Seeing what people build is the best motivation
Please feel free to get in touch through GitHub Issues.
VisualFries is licensed under the MIT License.