OmniAvatar

Batteries-included package for AI avatars with two surfaces:

live — real-time streaming avatars (LiveKit sessions, PCM audio streaming for lip-sync) for conversational agents
render — asynchronous batch avatar video generation (narration audio in, talking-head MP4 out) for offline pipelines such as presentation videos

Provides provider implementations for HeyGen, Tavus, bitHuman, and local rendering via LivePortrait + JoyVASA (Apple Silicon).

For core interfaces only (no provider dependencies), see omniavatar-core.

Quick Start (live)

import (
"github.com/plexusone/omniavatar""github.com/plexusone/omniavatar-core/live"
_ "github.com/plexusone/omniavatar/providers/all"
)
funcmain() {
provider, err:=omniavatar.GetLiveProvider("heygen",
omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
omniavatar.WithExtension("avatar_id", avatarID),
omniavatar.WithExtension("sandbox", true))
iferr!=nil {
log.Fatal(err)
}
session, err:=provider.CreateSession(live.SessionConfig{
AudioConfig: live.DefaultAudioConfig(),
})
iferr!=nil {
log.Fatal(err)
}
// Start with LiveKiterr=session.Start(ctx, &omniavatar.LiveKitStartOptions{
Room: room,
AgentIdentity: "agent-123",
LiveKitURL: os.Getenv("LIVEKIT_URL"),
LiveKitAPIKey: os.Getenv("LIVEKIT_API_KEY"),
LiveKitAPISecret: os.Getenv("LIVEKIT_API_SECRET"),
})
}

Quick Start (render)

import (
"github.com/plexusone/omniavatar""github.com/plexusone/omniavatar-core/render"
_ "github.com/plexusone/omniavatar/providers/all"
)
funcmain() {
provider, err:=omniavatar.GetRenderProvider("bithuman",
omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
omniavatar.WithExtension("agent_id", agentID))
iferr!=nil {
log.Fatal(err)
}
// Providers with hosting support can upload local narration audio.audioURL:=""ifup, ok:=provider.(render.AudioUploader); ok {
audioURL, err=up.UploadAudio(ctx, "narration.mp3", audioFile)
iferr!=nil {
log.Fatal(err)
}
}
job, err:=provider.Generate(ctx, render.GenerateRequest{
AvatarID: agentID,
AudioURL: audioURL,
})
iferr!=nil {
log.Fatal(err)
}
status, err:=render.Wait(ctx, provider, job.ID, 5*time.Second)
iferr!=nil {
log.Fatal(err)
}
log.Printf("video ready: %s (%.1fs)", status.VideoURL, status.Duration)
out, err:=os.Create("presenter.mp4")
iferr!=nil {
log.Fatal(err)
}
deferout.Close()
iferr:=provider.Download(ctx, job.ID, out); err!=nil {
log.Fatal(err)
}
}

Architecture

Adapters follow the PlexusOne convention: render adapters live in each provider SDK repo (heygen-go/omniavatar, …), depending only on omniavatar-core, so provider-specific knowledge stays with the SDK. The live adapters live here in the batteries-included package, because their LiveKit integration (LiveKitStartOptions, token generation) lives here. This package re-exports both, registered by name via providers/all.

omniavatar-core/ # Core interfaces + shared helpers (no provider deps)
├── live/ # Real-time session interfaces
├── render/ # Batch generation: Provider, AudioUploader,
│ # AvatarLister, GenerateRequest, Wait,
│ # AudioContentType/DownloadURL helpers
└── registry/ # Factory types
heygen-go/omniavatar/ # HeyGen RENDER adapter (core-only) — in the SDK repo
tavus-go/omniavatar/ # Tavus RENDER adapter
bithuman-go/omniavatar/ # bitHuman RENDER adapter
omniavatar/ # Batteries-included (this package)
├── registry.go # Global live + render registries
├── token.go / start_options.go # LiveKit token + start options
└── providers/
├── heygen/ # HeyGen LIVE adapter (LiveAvatar); registers the SDK render adapter
├── tavus/ # Tavus LIVE adapter (CVI)
├── bithuman/ # bitHuman LIVE adapter
├── liveportrait-joyvasa/ # Local RENDER adapter (Apple Silicon); registers core provider
└── all/ # Convenience import (registers every provider)

Provider Registry

Priority System

Providers register with a priority level:

Priority	Constant	Description
0	`PriorityThin`	Minimal implementations
10	`PriorityThick`	Full SDK implementations

Higher priority providers override lower priority registrations for the same name.

Auto-Registration

Providers auto-register both surfaces via init() when imported:

// Import specific providerimport _ "github.com/plexusone/omniavatar/providers/heygen"// Or import all providersimport _ "github.com/plexusone/omniavatar/providers/all"

Registry Functions

// Live (real-time sessions)provider, err:=omniavatar.GetLiveProvider("heygen", opts...)
names:=omniavatar.ListLiveProviders()
ok:=omniavatar.HasLiveProvider("heygen")
// Render (batch video generation)provider, err:=omniavatar.GetRenderProvider("heygen", opts...)
names:=omniavatar.ListRenderProviders()
ok:=omniavatar.HasRenderProvider("heygen")

Supported Providers

HeyGen

Live: real-time avatar with lip-sync using HeyGen LiveAvatar LITE mode. Render: HeyGen Video Generation API (v2).

Note: the live surface uses the LiveAvatar API key (LIVEAVATAR_API_KEY); the render surface uses the HeyGen API key (HEYGEN_API_KEY). They are different credentials.

// Liveprovider, err:=omniavatar.GetLiveProvider("heygen",
omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
omniavatar.WithExtension("avatar_id", "josh_lite3_20230714"),
omniavatar.WithExtension("sandbox", true), // 60s limit, no creditsomniavatar.WithExtension("video_quality", "high"), // very_high, high, medium, low
)
// Renderprovider, err:=omniavatar.GetRenderProvider("heygen",
omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
omniavatar.WithExtension("avatar_id", avatarID),
)

Surface	Option	Description
live	`avatar_id`	Avatar UUID (required)
live	`sandbox`	Enable sandbox mode (recommended for dev)
live	`video_quality`	Video quality preset
render	`avatar_id`	Default avatar ID
render	`upload_base_url`	Custom asset upload service URL (default: upload.heygen.com)
render request	`talking_photo_id`	Use a talking photo instead of an avatar
render request	`avatar_style`	normal, circle, closeUp
render request	`voice_id`	TTS voice for Script input
render request	`test`	Watermarked test video, no credits

The HeyGen render provider implements render.AudioUploader via the HeyGen asset upload API (MP3/audio/mpeg is the documented audio asset type).

Tavus

Live: real-time avatar using Tavus PAL (Personalized AI Likeness). Render: Tavus Video Generation using replicas.

// Liveprovider, err:=omniavatar.GetLiveProvider("tavus",
omniavatar.WithAPIKey(os.Getenv("TAVUS_API_KEY")),
omniavatar.WithExtension("pal_id", "pal_xxx"), // Optionalomniavatar.WithExtension("face_id", "face_xxx"), // Optional
)
// Renderprovider, err:=omniavatar.GetRenderProvider("tavus",
omniavatar.WithAPIKey(os.Getenv("TAVUS_API_KEY")),
omniavatar.WithExtension("replica_id", "rep_xxx"),
)

Surface	Option	Description
live	`pal_id`	PAL ID (optional, uses stock avatar if not set)
live	`face_id`	Face override (optional)
render	`replica_id`	Default replica ID
render request	`fast`	Faster generation (disables some features)
render request	`callback_url`	Completion webhook URL

Tavus has no audio upload API; supply a publicly fetchable GenerateRequest.AudioURL (.wav or .mp3).

bitHuman

Live: ultra-low latency real-time avatars. Render: bitHuman video generation, including audio upload support (render.AudioUploader).

// Liveprovider, err:=omniavatar.GetLiveProvider("bithuman",
omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
omniavatar.WithExtension("agent_id", "agent_xxx"),
)
// Renderprovider, err:=omniavatar.GetRenderProvider("bithuman",
omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
omniavatar.WithExtension("agent_id", "agent_xxx"),
)

Surface	Option	Description
live	`agent_id`	bitHuman agent ID (required)
render	`agent_id`	Default agent ID
render request	`voice_id`	TTS voice for Script input

LivePortrait + JoyVASA (Local)

Render-only: on-device audio-driven talking-head video generation on Apple Silicon. No cloud API required — connects to a local Python gRPC server.

// Render (no API key needed)provider, err:=omniavatar.GetRenderProvider("liveportrait-joyvasa")
// Upload local audio (returns local:// URL)audioURL, _:=provider.(render.AudioUploader).UploadAudio(ctx, "narration.wav", f)
job, _:=provider.Generate(ctx, render.GenerateRequest{
AvatarID: "john", // avatar bundle name in ~/.omniavatar/avatars/AudioURL: audioURL,
Extensions: map[string]any{
"seed": int64(42), // deterministic output"motion_scale": float32(1.2), // adjust expressiveness
},
})

Option	Description
`endpoint`	Custom Unix socket path (default: `/tmp/omniavatar-liveportrait-joyvasa.sock`)
`seed`	Random seed for deterministic output
`motion_scale`	Facial movement intensity (default: 1.0)

Setup: The provider requires a running Python server. See the omniavatar-core local render guide for server setup and avatar bundle format.

Performance: ~5 min for 13.7s output at 512×512 on Apple Silicon (M-series).

Session Lifecycle (live)

1. Get Provider → omniavatar.GetLiveProvider("heygen", opts...)
2. Create Session → provider.CreateSession(cfg)
3. Start → session.Start(ctx, &LiveKitStartOptions{...})
4. Wait for Join → session.WaitForJoin(ctx, 30*time.Second)
5. Stream Audio → session.AudioOutput().CaptureFrame(ctx, pcm)
6. Close → session.Close(ctx)

Job Lifecycle (render)

1. Get Provider → omniavatar.GetRenderProvider("heygen", opts...)
2. Upload Audio → provider.(render.AudioUploader).UploadAudio(...) [optional]
3. Generate → provider.Generate(ctx, render.GenerateRequest{...})
4. Wait → render.Wait(ctx, provider, job.ID, interval)
5. Download → provider.Download(ctx, job.ID, dst)

LiveKit Integration

Token Generation

Generate tokens for avatar participants to join LiveKit rooms:

token, err:=omniavatar.GenerateAvatarToken(omniavatar.TokenOptions{
APIKey: os.Getenv("LIVEKIT_API_KEY"),
APISecret: os.Getenv("LIVEKIT_API_SECRET"),
RoomName: "my-room",
Identity: "avatar-heygen-abc123",
Provider: "heygen",
AgentIdentity: "agent-123",
TTL: time.Hour,
})

Start Options

typeLiveKitStartOptionsstruct {
Room*lksdk.Room// LiveKit room referenceAgentIdentitystring// Agent's participant identityLiveKitURLstring// LiveKit server URLLiveKitAPIKeystring// API key for token generationLiveKitAPISecretstring// API secret for token generation
}

Audio Format (live)

Default audio configuration:

Parameter	Value
Sample Rate	24000 Hz
Channels	1 (mono)
Encoding	PCM16 (linear16)

Provider Comparison

Provider	Live Latency	Video Quality	Voice Cloning	Render Audio Upload
HeyGen	~500ms	Excellent	Yes	Yes (asset API, MP3)
Tavus	~300ms	Excellent	Yes (via PAL)	No (URL only)
bitHuman	~200ms	Good	No	Yes

Name		Name	Last commit message	Last commit date
.github		.github
docs		docs
examples/render-basic		examples/render-basic
providers		providers
.gitignore		.gitignore
.golangci.yaml		.golangci.yaml
CHANGELOG.json		CHANGELOG.json
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
doc.go		doc.go
go.mod		go.mod
go.sum		go.sum
mkdocs.yml		mkdocs.yml
registry.go		registry.go
registry_test.go		registry_test.go
start_options.go		start_options.go
token.go		token.go

Navigation Menu

Uh oh!

Repository files navigation

OmniAvatar

Quick Start (live)

Quick Start (render)

Architecture

Provider Registry

Priority System

Auto-Registration

Registry Functions

Supported Providers

HeyGen

Tavus

bitHuman

LivePortrait + JoyVASA (Local)

Session Lifecycle (live)

Job Lifecycle (render)

LiveKit Integration

Token Generation

Start Options

Audio Format (live)

Provider Comparison

Specs

Resources

About

Releases

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

OmniAvatar

Quick Start (live)

Quick Start (render)

Architecture

Provider Registry

Priority System

Auto-Registration

Registry Functions

Supported Providers

HeyGen

Tavus

bitHuman

LivePortrait + JoyVASA (Local)

Session Lifecycle (live)

Job Lifecycle (render)

LiveKit Integration

Token Generation

Start Options

Audio Format (live)

Provider Comparison

Specs

Resources

About

Resources

Stars

Watchers

Forks

Releases

Contributors

Languages