Avatar API Reference¶

The avatar package provides infrastructure for integrating avatars with voice agents, supporting both static images and live lip-sync providers.

Package Structure¶

avatar/
├── factory.go      # Unified factory (NewSession, Setup)
├── setup.go        # High-level setup helper
├── session.go      # Session interface
├── audio.go        # AudioDestination interface
├── datastream.go   # DataStreamAudioOutput
├── queue.go        # QueueAudioOutput (testing)
├── token.go        # Token generation
├── errors.go       # Error types
└── tavus/          # Tavus provider
    ├── client.go   # API client
    ├── session.go  # Session implementation
    └── register.go # Auto-registration

Factory (Recommended)¶

The factory provides a unified way to create avatar sessions regardless of provider.

Setup¶

High-level helper that handles both static images and live avatars.

func Setup(cfg SetupConfig) (*SetupResult, error)

SetupConfig:

Field	Type	Description
`Provider`	`string`	Avatar type: `"tavus"`, `"static"`, `""` (none)
`StaticImage`	`StaticImageConfig`	Static image settings
`Tavus`	`TavusConfig`	Tavus provider settings
`LiveKitURL`	`string`	LiveKit server URL (for live avatars)
`LiveKitAPIKey`	`string`	LiveKit API key (for live avatars)
`LiveKitAPISecret`	`string`	LiveKit API secret (for live avatars)

SetupResult:

Field	Type	Description
`Mode`	`SetupMode`	`SetupModeNone`, `SetupModeStatic`, or `SetupModeLive`
`Session`	`Session`	Live avatar session (nil for static/none)
`StaticImage`	`*StaticImageResult`	Static image config (nil for live/none)

Example:

import (
    "github.com/plexusone/omni-livekit/avatar"
    _ "github.com/plexusone/omni-livekit/avatar/tavus" // Register provider
)

result, err := avatar.Setup(avatar.SetupConfig{
    Provider: avatar.ProviderTavus,
    Tavus: avatar.TavusConfig{
        APIKey: os.Getenv("TAVUS_API_KEY"),
    },
    LiveKitURL:       os.Getenv("LIVEKIT_URL"),
    LiveKitAPIKey:    os.Getenv("LIVEKIT_API_KEY"),
    LiveKitAPISecret: os.Getenv("LIVEKIT_API_SECRET"),
})

switch result.Mode {
case avatar.SetupModeNone:
    // Audio-only, no avatar
case avatar.SetupModeStatic:
    // Configure agent with result.StaticImage
case avatar.SetupModeLive:
    // Use result.Session for live avatar
    defer result.Session.Close(ctx)
}

NewSession¶

Low-level factory that creates a Session for registered providers.

func NewSession(cfg Config) (Session, error)

Config:

Field	Type	Description
`Provider`	`string`	Provider name: `"tavus"`, etc.
`Tavus`	`TavusConfig`	Tavus-specific config
`Anam`	`AnamConfig`	Anam-specific config (future)
`Simli`	`SimliConfig`	Simli-specific config (future)

Example:

session, err := avatar.NewSession(avatar.Config{
    Provider: avatar.ProviderTavus,
    Tavus: avatar.TavusConfig{
        APIKey: os.Getenv("TAVUS_API_KEY"),
        PalID:  "your-pal-id",
    },
})

Provider Registration¶

Providers register themselves via init():

// In avatar/tavus/register.go
func init() {
    avatar.RegisterProvider("tavus", func(cfg avatar.Config) (avatar.Session, error) {
        return NewSession(SessionConfig{...})
    })
}

Import the provider package to register it:

import _ "github.com/plexusone/omni-livekit/avatar/tavus"

Provider Constants¶

Constant	Value	Description
`ProviderNone`	`""`	No avatar (audio-only)
`ProviderStatic`	`"static"`	Static image (not Session-based)
`ProviderTavus`	`"tavus"`	Tavus live avatar
`ProviderAnam`	`"anam"`	Anam avatar (future)
`ProviderSimli`	`"simli"`	Simli avatar (future)

Core Interfaces¶

Session¶

// Session manages a lip-sync avatar lifecycle.
type Session interface {
    // Start initializes the avatar session.
    Start(ctx context.Context, opts StartOptions) error

    // Stop ends the avatar session.
    Stop() error

    // AudioDestination returns the audio output for streaming to the avatar.
    AudioDestination() AudioDestination
}

AudioDestination¶

// AudioDestination receives audio frames for avatar lip-sync.
type AudioDestination interface {
    // Write sends PCM16 audio samples to the avatar.
    Write(samples []int16) error

    // Flush signals end of current speech segment.
    Flush() error

    // ClearBuffer interrupts current playback (for user interruption).
    ClearBuffer() error

    // Close releases resources.
    Close() error
}

Token Generation¶

GenerateAvatarToken¶

Generates a LiveKit JWT token for avatar participants.

func GenerateAvatarToken(cfg TokenConfig) (string, error)

TokenConfig:

Field	Type	Description
`APIKey`	`string`	LiveKit API key
`APISecret`	`string`	LiveKit API secret
`RoomName`	`string`	Room to join
`AvatarID`	`string`	Avatar participant identity
`AvatarName`	`string`	Avatar display name
`OnBehalfOf`	`string`	Agent identity (sets `lk.publish_on_behalf`)
`TTL`	`time.Duration`	Token validity duration

Example:

token, err := avatar.GenerateAvatarToken(avatar.TokenConfig{
    APIKey:     os.Getenv("LIVEKIT_API_KEY"),
    APISecret:  os.Getenv("LIVEKIT_API_SECRET"),
    RoomName:   "my-room",
    AvatarID:   "tavus-avatar",
    AvatarName: "AI Assistant",
    OnBehalfOf: "ai-agent",
    TTL:        time.Hour,
})

Audio Outputs¶

DataStreamAudioOutput¶

Streams audio to a remote avatar via LiveKit data streams.

func NewDataStreamAudioOutput(cfg DataStreamConfig) *DataStreamAudioOutput

DataStreamConfig:

Field	Type	Description
`Room`	`*lksdk.Room`	LiveKit room
`DestinationIdentity`	`string`	Avatar participant identity
`SampleRate`	`int`	Audio sample rate (default: 24000)

Example:

output := avatar.NewDataStreamAudioOutput(avatar.DataStreamConfig{
    Room:                room,
    DestinationIdentity: "tavus-avatar",
    SampleRate:          24000,
})

// Stream audio
output.Write(samples)
output.Flush()

// Handle interruption
output.ClearBuffer()

QueueAudioOutput¶

In-memory audio queue for testing without network.

func NewQueueAudioOutput() *QueueAudioOutput

Methods:

Method	Description
`Write(samples)`	Enqueue audio samples
`Flush()`	Mark end of segment
`ClearBuffer()`	Clear queued audio
`Read() []int16`	Dequeue audio (for testing)
`Len() int`	Number of queued samples

Example:

output := avatar.NewQueueAudioOutput()

// Simulate TTS
output.Write([]int16{1, 2, 3})
output.Flush()

// Verify in tests
assert.Equal(t, 3, output.Len())

Error Types¶

Sentinel Errors¶

Error	Description
`ErrInvalidConfig`	Missing required configuration
`ErrSessionNotStarted`	Operation requires started session
`ErrAvatarJoinTimeout`	Avatar didn't join in time

ProviderError¶

Wraps errors from avatar providers (Tavus, etc.).

type ProviderError struct {
    Provider  string  // e.g., "tavus"
    Operation string  // e.g., "create_conversation"
    Err       error   // Underlying error
}

Example:

var providerErr *avatar.ProviderError
if errors.As(err, &providerErr) {
    log.Printf("Provider %s failed on %s: %v",
        providerErr.Provider,
        providerErr.Operation,
        providerErr.Unwrap())
}

Tavus Provider¶

tavus.Client¶

API client for Tavus CVI (Conversational Video Interface).

func NewClient(cfg ClientConfig) (*Client, error)

ClientConfig:

Field	Type	Description
`APIKey`	`string`	Tavus API key (required)
`BaseURL`	`string`	API base URL (default: https://tavusapi.com)
`HTTPClient`	`*http.Client`	Custom HTTP client

CreateConversation¶

Creates a new avatar conversation.

func (c *Client) CreateConversation(ctx context.Context, req CreateConversationRequest) (*CreateConversationResponse, error)

CreateConversationRequest:

Field	Type	Description
`PalID`	`string`	PAL to use (default: `DefaultPalID`)
`FaceID`	`string`	Optional face override
`LiveKitURL`	`string`	LiveKit WebSocket URL
`LiveKitToken`	`string`	JWT token for avatar
`ConversationName`	`string`	Optional name

CreateConversationResponse:

Field	Type	Description
`ConversationID`	`string`	Unique conversation ID
`ConversationURL`	`string`	Join URL (if available)

CreatePal¶

Creates a new PAL (Personality AI Likeness).

func (c *Client) CreatePal(ctx context.Context, req CreatePalRequest) (*CreatePalResponse, error)

CreatePalRequest:

Field	Type	Description
`PalName`	`string`	Display name
`DefaultFaceID`	`string`	Face ID (required)
`PipelineMode`	`string`	Processing mode (default: "echo")
`TransportType`	`string`	Transport type (default: "livekit")

EndConversation¶

Ends an active conversation.

func (c *Client) EndConversation(ctx context.Context, conversationID string) error

SDK¶

Returns the underlying tavus-go SDK client for advanced usage.

func (c *Client) SDK() *tavussdk.Client

Constants¶

Constant	Value	Description
`tavus.DefaultPalID`	`"pb87e71797da"`	Stock Tavus PAL for testing

Avatar API Reference¶

Package Structure¶

Factory (Recommended)¶

Setup¶

NewSession¶

Provider Registration¶

Provider Constants¶

Core Interfaces¶

Session¶

AudioDestination¶

Token Generation¶

GenerateAvatarToken¶

Audio Outputs¶

DataStreamAudioOutput¶

QueueAudioOutput¶

Error Types¶

Sentinel Errors¶

ProviderError¶

Tavus Provider¶

tavus.Client¶

CreateConversation¶

CreatePal¶

EndConversation¶

SDK¶

Constants¶

See Also¶