Skip to content

v0.3.0

Release Date: 2026-07-27

Adds the local render provider — on-device audio-driven talking-head video generation on Apple Silicon, no cloud API required.

Highlights

  • Local render provider: providers/liveportrait-joyvasa — the first on-device engine, using LivePortrait + JoyVASA (MIT/Apache, fully-free)

Added

  • providers/liveportrait-joyvasa — local render provider connecting to a Python gRPC server on Apple Silicon (MPS)
  • Local provider registered in providers/all for auto-discovery via: 
    import _ "github.com/plexusone/omniavatar/providers/all"
provider, _ := omniavatar.GetRenderProvider("liveportrait-joyvasa")

Usage

import (
    "github.com/plexusone/omniavatar"
    "github.com/plexusone/omniavatar-core/render"
    _ "github.com/plexusone/omniavatar/providers/liveportrait-joyvasa"
)

// No API key needed for local provider
provider, _ := omniavatar.GetRenderProvider("liveportrait-joyvasa")

// Upload local audio (returns local:// URL)
audioURL, _ := provider.(render.AudioUploader).UploadAudio(ctx, "narration.wav", f)

job, _ := provider.Generate(ctx, render.GenerateRequest{
    AvatarID: "john",  // avatar bundle name
    AudioURL: audioURL,
})

status, _ := render.Wait(ctx, provider, job.ID, 3*time.Second)
provider.Download(ctx, job.ID, outFile)

Setup

The provider requires a running Python server. Start it with:

cd $GOPATH/src/github.com/plexusone/omniavatar-core/providers/liveportrait-joyvasa/server
./run.sh

See the omniavatar-core local render guide for full setup instructions and avatar bundle format.

Performance

On Apple Silicon (M-series):

Metric Value
Resolution 512×512
Speed ~5 min for 13.7s output
Memory ~4GB with model loaded

Full Changelog | Compare