Local Avatar Render Providers — Spec¶
Design specs for local, offline avatar render providers in OmniAvatar:
talking-head video generated on-device (no cloud API, no per-minute cost, audio
never leaves the machine), exposed through the existing render.Provider
interface so videoascode selects them like the HeyGen/Tavus/bitHuman providers.
This mirrors the local F5-TTS / Whisper providers in omnivoice-core: a thin Go
gRPC client per model over a Unix Domain Socket, backed by a persistent
Python/MLX worker.
Documents¶
| Doc | Purpose |
|---|---|
| Engine Landscape | Which engines can back the provider — commercial × local-MPS matrix and the decision |
| PRD | Product requirements — why, goals, non-goals, success criteria |
| TRD | Technical design — interface mapping, transport, decisions D1–D3 |
| PLAN | Execution narrative — critical path, sequencing, risks, rollout |
| ROADMAP | Tracked checklist — phases and RMIs (RMI-OACORE-NNN) |
Key decisions (see TRD)¶
- D1 — Providers live in
omniavatar-core/providers/<engine>/, one package per complete engine, each with its own Pythonserver/(venv + socket). Mirrorsomnivoice-core/providers/{f5tts-mlx,whisper-mlx}; isolates conflicting Python dependency stacks. - D3 — Named per model (
latentsync,echomimic, …), like the cloud providers. LivePortrait is a motion pre-stage inside the LatentSync provider, not its own provider.
Status¶
Draft. Execution is gated on RMI-OACORE-001 (the Apple-Silicon engine spike):
no Go work begins until a runnable engine is chosen. Spike findings are recorded
in spike/ as they are produced.
Supersedes the exploratory IDEATION_CHAT_LOCAL.md (repo root), which predates
the current interfaces.