OmniAvatar Core

Core interfaces for AI avatars. This package provides the interface definitions without any provider dependencies, across two surfaces:

• live — real-time streaming avatar sessions (rooms, PCM audio streaming for lip-sync) for conversational agents
• render — asynchronous batch avatar video generation (narration audio in, talking-head MP4 out) for offline pipelines

For a batteries-included package with all providers, see omniavatar.

Architecture

Adapters follow the PlexusOne convention: render adapters live in each provider SDK repo (depending only on this module), while live adapters live in the batteries-included omniavatar package (their LiveKit integration lives there).

omniavatar-core/ # Core interfaces + shared helpers (no provider deps)
├── live/ # Real-time sessions
│ ├── provider.go # Provider interface
│ ├── session.go # Session interface + callbacks
│ ├── audio.go # AudioDestination interface
│ └── errors.go # Error types
├── render/ # Batch video generation
│ ├── provider.go # Provider + AudioUploader interfaces
│ ├── lister.go # AvatarLister + AvatarInfo
│ ├── request.go # GenerateRequest
│ ├── job.go # Job, JobState, JobStatus, Wait
│ ├── mediatype.go / download.go # AudioContentType / DownloadURL helpers
│ └── errors.go # Error types
└── registry/
└── registry.go # Factory types + options
heygen-go/omniavatar/ # HeyGen RENDER adapter (core-only) — in the SDK repo
tavus-go/omniavatar/ # Tavus RENDER adapter
bithuman-go/omniavatar/ # bitHuman RENDER adapter
omniavatar/ # Batteries-included
├── registry.go # Global live + render registries
├── token.go / start_options.go # LiveKit token + start options
└── providers/
├── heygen/ # HeyGen LIVE adapter; registers the SDK render adapter
├── tavus/ # Tavus LIVE adapter
├── bithuman/ # bitHuman LIVE adapter
└── all/ # Convenience import

Live Interfaces

Provider

Creates avatar sessions with provider-specific configuration.

typeProviderinterface {
Name() stringCreateSession(cfgSessionConfig) (Session, error)
}

Session

Manages the avatar lifecycle: start, audio streaming, and cleanup.

typeSessioninterface {
Identity() stringProvider() stringStart(ctx context.Context, optsany) errorWaitForJoin(ctx context.Context, timeout time.Duration) errorAudioOutput() AudioDestinationClose(ctx context.Context) errorSetCallbacks(callbacks*SessionCallbacks)
}

AudioDestination

Streams TTS audio to the avatar for lip-sync playback.

typeAudioDestinationinterface {
CaptureFrame(ctx context.Context, frame []byte) errorFlush(ctx context.Context) errorClearBuffer(ctx context.Context) errorSampleRate() intChannels() intClose() error
}

Render Interfaces

Provider

Generates avatar videos asynchronously: submit, poll, download.

typeProviderinterface {
Name() stringGenerate(ctx context.Context, reqGenerateRequest) (*Job, error)
Status(ctx context.Context, jobIDstring) (*JobStatus, error)
Download(ctx context.Context, jobIDstring, dst io.Writer) error
}

AudioUploader (optional capability)

Providers that can host local audio files implement this; feature-detect it:

ifup, ok:=provider.(render.AudioUploader); ok {
audioURL, err=up.UploadAudio(ctx, "narration.mp3", f)
}

AvatarLister (optional capability)

Providers that can enumerate the account's avatars implement this; the returned AvatarInfo.ID values are directly usable as AvatarID:

ifl, ok:=provider.(render.AvatarLister); ok {
avatars, err:=l.ListAvatars(ctx, "abigail") // "" = all
}

GenerateRequest

job, err:=provider.Generate(ctx, render.GenerateRequest{
AvatarID: avatarID, // heygen avatar_id / tavus replica_id / bithuman agent_idAudioURL: narrationURL, // drives lip-sync from existing audio (primary path)
})

Wait

status, err:=render.Wait(ctx, provider, job.ID, 5*time.Second)
// status.State: pending → processing → completed | failed

Usage

Import only the interfaces:

import (
"github.com/plexusone/omniavatar-core/live""github.com/plexusone/omniavatar-core/render"
)
funcprocessAvatar(session live.Session) error {
audioOut:=session.AudioOutput()
returnaudioOut.CaptureFrame(ctx, pcmData)
}

Import with all providers (batteries-included):

import (
"github.com/plexusone/omniavatar"
_ "github.com/plexusone/omniavatar/providers/all"
)
liveProvider, err:=omniavatar.GetLiveProvider("heygen",
omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
omniavatar.WithExtension("avatar_id", avatarID),
omniavatar.WithExtension("sandbox", true),
)
renderProvider, err:=omniavatar.GetRenderProvider("heygen",
omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
)

Provider Registry Pattern

Providers register with a priority level:

Priority	Constant	Description
0	PriorityThin	Minimal implementations
10	PriorityThick	Full SDK implementations

Higher priority providers override lower priority registrations for the same name.

Registration via init()

The batteries omniavatar package registers providers by name via init(). The render provider is the SDK-hosted adapter; the live provider is local:

// In omniavatar/providers/heygen/register.goimport heygenrender "github.com/plexusone/heygen-go/omniavatar"funcinit() {
omniavatar.RegisterLiveProvider("heygen", NewProviderFromConfig, omniavatar.PriorityThick)
omniavatar.RegisterRenderProvider("heygen", heygenrender.NewRenderProviderFromConfig, omniavatar.PriorityThick)
}

Supported Providers

Provider	Live	Render	Live Latency
HeyGen	LiveAvatar LITE mode	Video Generation v2	~500ms
Tavus	Conversational Video	Video Generation (replicas)	~300ms
bitHuman	Real-time Avatars	Video Generation + audio upload	~200ms

Local Render Providers

Local avatar rendering — talking-head video generated on-device behind the same render.Provider interface, no cloud API required.

Available:providers/liveportrait-joyvasa — the first local engine.

Engine	License	Device	Resolution	Speed
LivePortrait + JoyVASA	MIT/Apache	Apple Silicon (MPS)	512×512	~5 min / 13.7s

import lp "github.com/plexusone/omniavatar-core/providers/liveportrait-joyvasa"provider, _:=lp.New("") // connects to local Python serveraudioURL, _:=provider.UploadAudio(ctx, "narration.wav", f)
job, _:=provider.Generate(ctx, render.GenerateRequest{
AvatarID: "john", // avatar bundle nameAudioURL: audioURL,
})

The provider connects to a Python gRPC server running the inference pipeline (HuBERT → DiT motion → LivePortrait render → ffmpeg encode). See the Local Avatar Render guide for setup.

See also: Local Avatar Render spec (PRD / TRD / Plan / Roadmap) and engine landscape.

Session Lifecycle (live)

1. Get Provider → omniavatar.GetLiveProvider("heygen", opts...)
2. Create Session → provider.CreateSession(cfg)
3. Start → session.Start(ctx, startOptions)
4. Wait for Join → session.WaitForJoin(ctx, timeout)
5. Stream Audio → session.AudioOutput().CaptureFrame(ctx, pcm)
6. Close → session.Close(ctx)

Job Lifecycle (render)

1. Get Provider → omniavatar.GetRenderProvider("heygen", opts...)
2. Upload Audio → provider.(render.AudioUploader).UploadAudio(...) [optional]
3. Generate → provider.Generate(ctx, render.GenerateRequest{...})
4. Wait → render.Wait(ctx, provider, job.ID, interval)
5. Download → provider.Download(ctx, job.ID, dst)

Session Callbacks (live)

session.SetCallbacks(&live.SessionCallbacks{
OnAvatarJoined: func(identitystring) {
log.Printf("Avatar joined: %s", identity)
},
OnPlaybackStarted: func() {
log.Print("Avatar started speaking")
},
OnPlaybackFinished: func(positionfloat64, interruptedbool) {
log.Printf("Avatar finished speaking at %.2fs (interrupted: %v)", position, interrupted)
},
OnError: func(errerror) {
log.Printf("Avatar error: %v", err)
},
})

Audio Format (live)

Default audio configuration for avatar providers:

Parameter	Value
Sample Rate	24000 Hz
Channels	1 (mono)
Encoding	PCM16 (linear16)

Resources

• omniavatar - Provider implementations
• HeyGen LiveAvatar
• HeyGen API
• Tavus
• bitHuman