Skip to content

OmniMeet Roadmap

PlexusOne OmniMeet — AI-native meeting and real-time collaboration abstraction for Go.

This document outlines the vision, architecture, and phased implementation plan for OmniMeet, the real-time collaboration layer of the PlexusOne ecosystem.

Table of Contents

Vision

OmniMeet provides a unified abstraction for real-time collaboration platforms—LiveKit, Daily, Zoom, Google Meet, Microsoft Teams, Discord Voice, and Slack Huddles—enabling AI agents to participate in meetings as first-class participants alongside humans.

Key insight: A meeting is not primarily about video. It's a live collaborative session containing participants, media streams, shared context, and real-time events. Video is just one capability.

OmniAgent
     │
     ├── OmniLLM      (reasoning)
     ├── OmniChat     (async messaging)
     ├── OmniVoice    (speech: TTS/STT)
     ├── OmniMemory   (conversation memory)
     └── OmniMeet     (real-time collaboration)  ← NEW
             │
             ├── omni-livekit
             ├── omni-daily
             └── omni-zoom

Goals

V1 Goals

  1. Meeting lifecycle management — Create, get, list, and end meetings
  2. Participant management — Join/leave humans and AI agents
  3. Audio-first — Real-time audio conversation between humans and agents
  4. Event-driven — Rich event stream (join, leave, track published, transcript updated)
  5. Provider-neutral — Same interface works across LiveKit, Daily, and eventually Zoom
  6. Agent participation — AI agents join meetings and interact via OmniVoice/OmniLLM
  7. Transcript capture — Real-time transcription via OmniVoice STT
  8. Recording hooks — Start/stop recording via provider APIs
  9. Attend external meetings — Join Zoom, Google Meet, and Teams meetings hosted by others, starting with phone dial-in (see Phase 3)

V1.5 Goals

  1. Video support — Optional video tracks for participants
  2. Screen share metadata — Detect and expose screen share events
  3. Data channels — In-meeting messaging and data exchange
  4. Invite integration — Generate join links, integrate with OmniChat for delivery

V2 Goals

  1. Scheduling — Calendar integration (Google, Outlook, CalDAV)
  2. Recurring meetings — Series management
  3. Breakout rooms — Sub-meetings within larger meetings

Non-Goals (V1)

The following are explicitly out of scope for V1:

  • Full calendar/scheduling system (→ future OmniCalendar)
  • Recurring meeting management
  • Complex RSVP workflows
  • Breakout rooms
  • Whiteboards / collaborative documents
  • Deep video understanding / computer vision
  • Native Teams and Webex connectors (Teams and Webex meetings are attended by dial-in)
  • WebRTC implementation details (delegated to providers)

Architecture Overview

Package Structure

github.com/plexusone/omnimeet-core/
├── meeting/           # Meeting type and lifecycle
├── participant/       # Participant types and management
├── track/             # Media tracks (audio, video, data)
├── recording/         # Recording abstraction
├── transcript/        # Transcript types (integrates with omnivoice)
├── event/             # Meeting events
├── provider/          # Provider interface and multi-provider client
├── registry/          # Provider factory registration
├── config/            # Configuration types
├── errors/            # Domain errors
└── docs/
    ├── specs/         # Specifications and roadmap
    └── adr/           # Architecture Decision Records

github.com/plexusone/omni-livekit/    # LiveKit provider
github.com/plexusone/omni-daily/      # Daily provider (depends on daily-go)
github.com/plexusone/daily-go/        # Daily REST API client
github.com/plexusone/omnimeet/        # Bundle with common providers

Dependency Graph

omnimeet (bundle)
    │
    ├── omnimeet-core (interfaces)
    │
    ├── omni-livekit
    │       └── livekit/server-sdk-go
    │
    └── omni-daily
            └── daily-go (REST client)
                    └── Daily REST API

Core Abstractions

Meeting

The primary abstraction representing a live collaborative session.

type Meeting struct {
    ID           string
    Name         string
    Status       MeetingStatus      // pending, active, ended
    Provider     string             // "livekit", "daily", "zoom"
    Participants []Participant
    CreatedAt    time.Time
    StartedAt    *time.Time
    EndedAt      *time.Time
    Metadata     map[string]string
}

type MeetingStatus string

const (
    MeetingStatusPending MeetingStatus = "pending"
    MeetingStatusActive  MeetingStatus = "active"
    MeetingStatusEnded   MeetingStatus = "ended"
)

Participant

An entity that joins a meeting—human, AI agent, recorder, or observer.

type Participant struct {
    ID         string
    MeetingID  string
    Kind       ParticipantKind    // human, agent, recorder, observer, sip
    Name       string
    Identity   string             // Provider-specific identity
    JoinedAt   time.Time
    LeftAt     *time.Time
    Tracks     []Track
    Metadata   map[string]string
}

type ParticipantKind string

const (
    ParticipantKindHuman    ParticipantKind = "human"
    ParticipantKindAgent    ParticipantKind = "agent"
    ParticipantKindRecorder ParticipantKind = "recorder"
    ParticipantKindObserver ParticipantKind = "observer"
    ParticipantKindSIP      ParticipantKind = "sip"
)

Track

A media stream published by a participant.

type Track struct {
    ID            string
    ParticipantID string
    Kind          TrackKind          // audio, video, screen_share, data
    Source        TrackSource        // microphone, camera, screen, unknown
    Muted         bool
    Metadata      map[string]string
}

type TrackKind string

const (
    TrackKindAudio       TrackKind = "audio"
    TrackKindVideo       TrackKind = "video"
    TrackKindScreenShare TrackKind = "screen_share"
    TrackKindData        TrackKind = "data"
)

MeetingProvider Interface

The core provider interface that all implementations must satisfy.

type MeetingProvider interface {
    // Provider identification
    Name() string

    // Meeting lifecycle
    CreateMeeting(ctx context.Context, req CreateMeetingRequest) (*Meeting, error)
    GetMeeting(ctx context.Context, meetingID string) (*Meeting, error)
    ListMeetings(ctx context.Context, opts ListMeetingsOptions) ([]Meeting, error)
    EndMeeting(ctx context.Context, meetingID string) error

    // Participant management
    CreateJoinToken(ctx context.Context, req JoinTokenRequest) (*JoinToken, error)
    RemoveParticipant(ctx context.Context, meetingID, participantID string) error
    ListParticipants(ctx context.Context, meetingID string) ([]Participant, error)

    // Events
    OnEvent(handler EventHandler)

    // Recording (optional capability)
    StartRecording(ctx context.Context, meetingID string, opts RecordingOptions) (*Recording, error)
    StopRecording(ctx context.Context, meetingID string) (*Recording, error)

    // Lifecycle
    Close() error
}

AgentParticipant Interface

For AI agents that actively participate in meetings.

type AgentParticipant interface {
    // Join as an agent participant
    JoinMeeting(ctx context.Context, meetingID string, token *JoinToken) error

    // Audio handling
    SubscribeToAudio(ctx context.Context, participantID string) (<-chan []byte, error)
    PublishAudio(ctx context.Context, audio []byte) error

    // Events
    OnTrackPublished(handler TrackHandler)
    OnParticipantJoined(handler ParticipantHandler)
    OnParticipantLeft(handler ParticipantHandler)

    // Data messages
    SendDataMessage(ctx context.Context, msg DataMessage) error
    OnDataMessage(handler DataMessageHandler)

    // Lifecycle
    LeaveMeeting(ctx context.Context) error
}

Events

Rich event stream for meeting lifecycle.

type Event struct {
    Type      EventType
    MeetingID string
    Timestamp time.Time
    Data      any
}

type EventType string

const (
    EventMeetingStarted       EventType = "meeting.started"
    EventMeetingEnded         EventType = "meeting.ended"
    EventParticipantJoined    EventType = "participant.joined"
    EventParticipantLeft      EventType = "participant.left"
    EventTrackPublished       EventType = "track.published"
    EventTrackUnpublished     EventType = "track.unpublished"
    EventTrackMuted           EventType = "track.muted"
    EventTrackUnmuted         EventType = "track.unmuted"
    EventTranscriptUpdated    EventType = "transcript.updated"
    EventRecordingStarted     EventType = "recording.started"
    EventRecordingStopped     EventType = "recording.stopped"
    EventDataMessageReceived  EventType = "data_message.received"
)

Provider Strategy

Provider Implementation Order

Order Provider Rationale
1 LiveKit Go-native, WebRTC SFU, strong agent SDK, ideal for Go-first architecture
2 Daily REST API + SDKs, different model validates abstraction, Pipecat ecosystem
3 Zoom SaaS integration, different concerns (scheduling, invitations, OAuth)
4 Agora Large-scale RTC, stress-tests abstraction at scale

Provider Comparison

Capability LiveKit Daily Zoom
Core model Room/Participant/Track Room/Participant Meeting/Participant
Go SDK First-class REST API (no Go SDK) REST API
Agent participation LiveKit Agents (Go) daily-python sidecar Meeting SDK
Self-hosted Yes No (SaaS) No (SaaS)
Recording Egress API REST API Cloud recording
SIP/PSTN Built-in Built-in Add-on

Implementation Approach

omni-livekit: - Full Go implementation using livekit/server-sdk-go - Agent participation via LiveKit Agents framework - Control plane + media plane in Go

omni-daily: - Create daily-go REST client for control plane - Use daily-js for browser participants (frontend) - Python sidecar optional for server-side agent participation

omni-zoom (future): - REST API for meeting management - Meeting SDK for bot attendance - OAuth2 for authentication

Attending External Meetings

The providers above host meetings the application controls. Attending a meeting hosted by someone else on Zoom, Google Meet, Teams, or Webex is a different problem: there is no join token to mint, and native bot access varies by platform. OmniMeet attends external meetings through connectors, added in this order (see ADR-009):

Order Connector Platforms What it provides
1 Phone dial-in Any meeting with a dial-in number Mixed audio in and out; no display name or speaker names; no platform app registration
2 Browser Google Meet, then the Zoom web client Visible participant with a display name; speaker names from live captions
3 Zoom Meeting SDK Zoom Only if the Zoom web client proves unreliable in the browser connector

Teams and Webex are attended by dial-in only. A native connector is reconsidered if either becomes common enough to need speaker names.

Every connector implements the same attendee session interface and reports its capabilities, so agents adapt to what the connection supports (for example, speaking but not reading chat).

Integration with PlexusOne Ecosystem

OmniVoice Integration

OmniMeet uses OmniVoice for speech processing within meetings.

Meeting
    │
    ├── Human speaks
    │       │
    │       ▼
    │   Agent subscribes to audio
    │       │
    │       ▼
    │   OmniVoice STT
    │       │
    │       ▼
    │   OmniLLM (reasoning)
    │       │
    │       ▼
    │   OmniVoice TTS
    │       │
    │       ▼
    │   Agent publishes audio
    │       │
    │       ▼
    └── Human hears response

OmniAgent Integration

OmniAgent orchestrates agent behavior during meetings.

// Agent joining a meeting
agent := omniagent.New(config)
meetingProvider := omnimeet.GetProvider("livekit")

meeting, _ := meetingProvider.CreateMeeting(ctx, omnimeet.CreateMeetingRequest{
    Name: "Sales Demo",
})

token, _ := meetingProvider.CreateJoinToken(ctx, omnimeet.JoinTokenRequest{
    MeetingID:   meeting.ID,
    Participant: omnimeet.ParticipantInfo{
        Name: "AI Sales Assistant",
        Kind: omnimeet.ParticipantKindAgent,
    },
})

// Agent joins and processes audio
agentParticipant := meetingProvider.CreateAgentParticipant()
agentParticipant.JoinMeeting(ctx, meeting.ID, token)

audioStream, _ := agentParticipant.SubscribeToAudio(ctx, humanParticipantID)
for audio := range audioStream {
    // STT → OmniAgent → TTS → PublishAudio
    transcript := omnivoice.Transcribe(ctx, audio)
    response := agent.Process(ctx, sessionID, transcript)
    audioResponse := omnivoice.Synthesize(ctx, response)
    agentParticipant.PublishAudio(ctx, audioResponse)
}

OmniChat Integration

Meeting invites can be delivered via OmniChat.

meeting, _ := meetingProvider.CreateMeeting(ctx, req)
joinLink := meetingProvider.GetJoinLink(meeting.ID)

// Send invite via Slack
slackProvider := omnichat.GetProvider("slack")
slackProvider.Send(ctx, channelID, omnichat.OutgoingMessage{
    Content: fmt.Sprintf("Join the meeting: %s", joinLink),
})

OmniMemory Integration

Meeting context and transcripts flow into memory.

// After meeting ends
transcript := meetingProvider.GetTranscript(ctx, meetingID)
memory.Store(ctx, omnimemory.StoreRequest{
    Content:  transcript.Text,
    Metadata: map[string]string{
        "type":       "meeting_transcript",
        "meeting_id": meetingID,
    },
})

Phased Roadmap

Phase 1: Foundation (V0.1) - COMPLETED

Goal: Core interfaces and LiveKit provider

Deliverables:

  • [x] omnimeet-core package structure
  • [x] Core types: Meeting, Participant, Track, Event
  • [x] MeetingProvider interface
  • [x] AgentParticipant interface
  • [x] Provider registry system
  • [x] omni-livekit implementation
  • [x] Meeting lifecycle (create, get, list, end)
  • [x] Join token generation
  • [x] Participant management
  • [x] Event streaming via callbacks
  • [x] Basic example: human + agent in LiveKit room
  • [ ] Webhook handler implementation

Dependencies: - livekit/server-sdk-go

Completed: 2026-07-03

Phase 2: Agent Participation (V0.2) - COMPLETED

Goal: Full agent participation in LiveKit meetings

Deliverables:

  • [x] LiveKit agent participant implementation
  • [x] Join meeting as agent
  • [x] Subscribe to participant audio (interface)
  • [x] Publish audio responses (interface)
  • [x] Data message send/receive
  • [x] Audio pipeline implementation
  • [x] Opus decoding for incoming audio
  • [x] Opus encoding for outgoing audio
  • [x] PCM16 frame handling
  • [x] OmniVoice integration
  • [x] STT pipeline for incoming audio
  • [x] TTS pipeline for outgoing audio
  • [x] VoiceAgentParticipant helper
  • [x] OmniAgent integration
  • [x] Agent processes meeting conversation
  • [x] Tool calling during meetings (MeetingSkill with 10 tools)
  • [x] VoiceMeetingSkill for STT/TTS integration
  • [x] Demo: "Talk to an OmniAgent in a LiveKit room"
  • [x] examples/omniagent-meeting demo application

Dependencies: - omnivoice-core (STT/TTS) - omniagent (agent runtime)

Completed: 2026-07-03

Phases 3–5 below are tracked as initiative INIT-OMNIMEET-001, which also covers work in omnivoice-core, omni-twilio, omni-telnyx, and omniagent. Its ROADMAP lists the roadmap items and dependencies.

Phase 3: Dial-in Attendance

Goal: Join any meeting that offers phone dial-in, audio only, with no platform app registration

Deliverables: attendee session interfaces, the dialin package with keypad templates for Zoom, Google Meet, and Teams, an invite parser, the phone dial-in joiner, and a simulated meeting for tests

Phase 4: Browser Connector

Goal: Join Google Meet and the Zoom web client as a visible participant with a display name

Deliverables: browser joiner core in a separate provider module, speaker-attributed transcript events, the Google Meet adapter, and the Zoom web-client adapter

Phase 5: Zoom Meeting SDK

Goal: Native Zoom attendance, built only if the Zoom web-client adapter proves unreliable

Deliverables: Zoom Meeting SDK joiner and on-behalf-of authorization for other accounts' meetings

Phase 6: Daily Provider (V0.3)

Goal: Second provider validates abstraction

Deliverables:

  • [ ] daily-go REST API client
  • [ ] Rooms API
  • [ ] Meeting tokens API
  • [ ] Participants API
  • [ ] Recordings API
  • [ ] Webhooks
  • [ ] omni-daily provider implementation
  • [ ] Meeting lifecycle
  • [ ] Join token generation
  • [ ] Participant management
  • [ ] Event streaming
  • [ ] Validate omnimeet-core abstraction against two providers
  • [ ] Demo: Same agent code works with LiveKit and Daily

Dependencies: - Daily REST API

Phase 7: Recording & Transcripts (V0.4)

Goal: Recording and transcript capture

Deliverables:

  • [ ] Recording abstraction
  • [ ] Start/stop recording
  • [ ] Recording status events
  • [ ] Recording retrieval
  • [ ] Transcript integration
  • [ ] Real-time transcript via OmniVoice STT
  • [ ] Store transcript via OmniStorage
  • [ ] Transcript events
  • [ ] LiveKit Egress integration
  • [ ] Daily recording API integration

Phase 8: Frontend SDK (V0.5)

Goal: TypeScript/React SDK for browser participants

Deliverables:

  • [ ] @plexusone/omnimeet-js - Core TypeScript library
  • [ ] @plexusone/omnimeet-react - React components
  • [ ] <MeetingRoom> component
  • [ ] <ParticipantTile> component
  • [ ] <Controls> component (mute, leave, screen share)
  • [ ] <Transcript> component
  • [ ] Provider adapters
  • [ ] LiveKit React integration
  • [ ] Daily React integration
  • [ ] OmniAgent embedded UI integration

Phase 9: Bundle & Polish (V1.0)

Goal: Production-ready release

Deliverables:

  • [ ] omnimeet bundle package
  • [ ] Comprehensive documentation
  • [ ] Example applications
  • [ ] CI/CD pipeline
  • [ ] Performance benchmarks
  • [ ] Security review

ADR Index

Architecture Decision Records are stored in /docs/adr/.

ADR Title Status
ADR-001 Meeting as Primary Abstraction Accepted
ADR-002 Provider Pattern Alignment with PlexusOne Accepted
ADR-003 LiveKit as First Provider Accepted
ADR-004 Daily as Second Provider Accepted
ADR-005 Agent Participation Architecture Accepted
ADR-006 Event-Driven Architecture Accepted
ADR-007 OmniVoice Integration Strategy Accepted
ADR-008 Frontend SDK Architecture Proposed
ADR-009 Dial-in First for External Meetings Accepted

Success Criteria

V1.0 Success Criteria

  1. Abstraction validation: Same OmniAgent code works with both LiveKit and Daily
  2. Latency: Agent response time < 500ms (excluding LLM latency)
  3. Reliability: 99.9% uptime for meeting sessions
  4. Developer experience: Create meeting + agent join in < 20 lines of code
  5. Documentation: Complete API reference and examples

Demo Scenario

1. Human joins LiveKit room via browser
2. AI agent joins same room
3. Human speaks: "What's on my calendar today?"
4. Agent hears via OmniMeet → OmniVoice STT
5. Agent reasons via OmniLLM with calendar tool
6. Agent responds via OmniVoice TTS → OmniMeet
7. Human hears: "You have a team standup at 10am and..."
8. Transcript is captured and stored
9. Meeting ends, recording is available

References