RTC Adapter

@spatius/avatarkit-rtc feeds motion data from a supported RTC room into AvatarKit for local rendering. In its managed path, it also creates the RTC client and plays the remote audio.

Package boundary

Package	Role
`@spatius/avatarkit`	Loads and renders the avatar in the browser.
`@spatius/avatarkit-rtc`	Connects AvatarKit to LiveKit or Agora RTC rooms.

The RTC Adapter is part of the Web SDK family. Native Agora clients use AvatarKitRTC on iOS or ai.spatius:avatarkit-rtc on Android.

Integration	Use the Web RTC Adapter?
LiveKit Agents Integration	Yes, with `LiveKitProvider`.
Agora Convo AI Integration	Yes, with `AgoraProvider`.
Direct Mode Integration	No. Use `@spatius/avatarkit`.
Backend Mode Integration	No. Feed the backend output into the core AvatarKit SDK.

Install

Install the peer for the provider your application uses.

LiveKit
Agora

npm install @spatius/avatarkit @spatius/avatarkit-rtc livekit-client

npm install @spatius/avatarkit @spatius/avatarkit-rtc agora-rtc-sdk-ng

Package compatibility

@spatius/avatarkit-rtc declares peer dependencies on AvatarKit and the selected RTC client. Install the packages together, and treat peer-dependency warnings as compatibility errors. For exact current ranges, check the package’s published peerDependencies instead of relying on a version copied into this page.

Runtime contract

Initialize AvatarKit in RTC mode before constructing AvatarPlayer.

import { AvatarSDK, DrivingServiceMode } from '@spatius/avatarkit'

await AvatarSDK.initialize(appId, {
  drivingServiceMode: DrivingServiceMode.rtc,
})

Create the AvatarView and wait for its onFirstRendering callback before constructing or connecting an AvatarPlayer. AvatarView initializes its renderer asynchronously; the constructor returning is not a readiness signal.

const avatarView = new AvatarView(avatar, container)
await new Promise<void>((resolve) => {
  avatarView.onFirstRendering = resolve
})

const player = new AvatarPlayer(new LiveKitProvider(), avatarView)
await player.connect(connection)

The adapter owns the RTC-to-renderer data flow. Do not manually call yieldAudioData() or yieldFramesData() while it is active.

Providers

import {
  AgoraProvider,
  AvatarPlayer,
  LiveKitProvider,
} from '@spatius/avatarkit-rtc'

const provider = new LiveKitProvider()
const player = new AvatarPlayer(provider, avatarView)

Provider	Connection config	Used by
`LiveKitProvider`	`LiveKitConnectionConfig`	LiveKit Agents
`AgoraProvider`	`AgoraConnectionConfig`	Agora Convo AI

`LiveKitProviderOptions`

Most applications use new LiveKitProvider() without options. Use these options only when the room publishes motion data under custom track names.

Option	Type	Description
`animationTrackName`	`string`	Exact publication track name to treat as the motion data track.
`animationTrackNamePattern`	`RegExp`	Pattern used to match motion data track names.

const provider = new LiveKitProvider({
  animationTrackNamePattern: /^spatius-motion-/,
})

Connection configuration

interface LiveKitConnectionConfig {
  url: string
  token: string
  roomName: string
}

interface AgoraConnectionConfig {
  appId: string
  channel: string
  token?: string
  uid?: number
}

Use short-lived room credentials returned by your backend. Never put a LiveKit API secret, Agora App Certificate, or Spatius API Key in the browser.

`AvatarPlayer`

Constructor

new AvatarPlayer(
  provider: RTCProvider,
  avatarView: AvatarView,
  options?: AvatarPlayerOptions,
)

`AvatarPlayerOptions`

Option	Type	Default	Description
`logLevel`	`'info' \| 'warning' \| 'error' \| 'none'`	`'warning'`	SDK log level.
`enableJitterBuffer`	`boolean`	`true`	Buffers and orders motion data for smoother playback.
`maxBufferDelayMs`	`number`	`80`	Maximum jitter-buffer delay when buffering is enabled.

Properties

Property	Type	Description
`isConnected`	`boolean`	Whether the player is connected or attached.
`sessionSummary`	SDK-owned object	Cumulative playback diagnostics for this player lifetime.

Treat the shape of sessionSummary as diagnostics data, not application state.

Methods

Method	Returns	Description
`prepareConnection(config)`	`Promise<void>`	Best-effort LiveKit connection pre-warm. Does not join the room.
`connect(config)`	`Promise<void>`	Create and connect the provider-owned RTC client.
`disconnect()`	`Promise<void>`	Disconnect the provider-owned client and release SDK-owned resources.
`attach(client)`	`Promise<void>`	Attach to a host-owned RTC client. See Host-owned RTC clients.
`detach()`	`Promise<void>`	Remove adapter listeners without disconnecting the host-owned client.
`startPublishing()`	`Promise<void>`	Request microphone access and publish it.
`stopPublishing()`	`Promise<void>`	Unpublish and release the SDK-owned microphone track.
`publishAudio(track)`	`Promise<void>`	Publish a supplied audio `MediaStreamTrack`.
`unpublishAudio()`	`Promise<void>`	Unpublish a supplied track without stopping it.
`reconnect()`	`Promise<void>`	Reconnect with the last managed connection config.
`getConnectionState()`	`string`	Return the provider connection state.
`getNativeClient()`	`unknown`	Return the underlying LiveKit Room or Agora client.
`on(event, handler)`	`void`	Add an event handler.
`off(event, handler)`	`void`	Remove the same handler reference.

Audio publishing

Use startPublishing() for the microphone or supply any browser audio track with publishAudio().

const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
const track = stream.getAudioTracks()[0]

await player.publishAudio(track)

// Later
await player.unpublishAudio()
track.stop()

unpublishAudio() does not stop a host-supplied track. The application that created it must release it.

State and events

The package exports ConnectionState:

enum ConnectionState {
  Disconnected = 'disconnected',
  Connecting = 'connecting',
  Connected = 'connected',
  Reconnecting = 'reconnecting',
  Failed = 'failed',
}

Event	Handler argument	Meaning
`'connected'`	None	A managed provider connected.
`'disconnected'`	None	A managed provider disconnected.
`'error'`	`Error`	The provider reported an error.
`'connection-state-changed'`	Connection state string	The provider state changed.
`'stalled'`	None	The motion data stream stalled and the avatar returned to idle.

const recover = () => {
  void player.reconnect()
}

player.on('stalled', recover)
player.off('stalled', recover)

Host-owned RTC clients

The default connect() path lets the adapter create and own the RTC client. Use attach() only when your application already owns that client.

LiveKit Room

Create the room with singlePeerConnection: false, attach before connecting, and use detach() before your application disconnects the room. In this flow, your application also owns remote audio playback.

import { Room } from 'livekit-client'
import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc'

const room = new Room({ singlePeerConnection: false })
const player = new AvatarPlayer(new LiveKitProvider(), avatarView)

await player.attach(room)
await room.connect(url, token)

// Later
await player.detach()
await room.disconnect()

Agora client

Attach before joining the channel. The host-owned client must use the H.264 codec and remains responsible for joining, subscribing, audio playback, and leaving. Do not mix connect() with attach() on the same player.

Native client access

For provider-specific features in the managed path, read the client owned by the provider:

import type { AgoraClient, LiveKitRoom } from '@spatius/avatarkit-rtc'

const room: LiveKitRoom | null = liveKitProvider.getNativeClient()
const agoraClient: AgoraClient | null = agoraProvider.getNativeClient()

player.getNativeClient() returns the same object as unknown; keep the provider reference when you want the provider-specific type.

Browser requirements

The package targets browser applications. AvatarKit’s WebAssembly and rendering requirements still apply; see Web Toolchain Setup. LiveKitProvider additionally requires RTCRtpScriptTransform to receive motion data. Check for the capability at runtime instead of relying on a fixed browser-version table:

const liveKitMotionSupported =
  typeof globalThis.RTCRtpScriptTransform !== 'undefined'

Browser autoplay policies can also require a user gesture before remote audio plays or microphone capture begins.

Reference Overview

Client SDKs

Server SDKs

Session Token API

Regions & Endpoints

Error Handling

Package boundary

Install

Package compatibility

Runtime contract

Providers

`LiveKitProviderOptions`

Connection configuration

`AvatarPlayer`

Constructor

`AvatarPlayerOptions`

Properties

Methods

Audio publishing

State and events

Host-owned RTC clients

LiveKit Room

Agora client

Native client access

Browser requirements

Next steps

LiveKit Agents Client

Agora Convo AI Client

​Package boundary

​Install

​Package compatibility

​Runtime contract

​Providers

​LiveKitProviderOptions

​Connection configuration

​AvatarPlayer

​Constructor

​AvatarPlayerOptions

​Properties

​Methods

​Audio publishing

​State and events

​Host-owned RTC clients

​LiveKit Room

​Agora client

​Native client access

​Browser requirements

​Next steps

LiveKit Agents Client

Agora Convo AI Client

Package boundary

Install

Package compatibility

Runtime contract

Providers

`LiveKitProviderOptions`

Connection configuration

`AvatarPlayer`

Constructor

`AvatarPlayerOptions`

Properties

Methods

Audio publishing

State and events

Host-owned RTC clients

LiveKit Room

Agora client

Native client access

Browser requirements

Next steps