AvatarKit Web SDK Reference

Quick Reference

Class	Purpose	Key methods
`AvatarSDK`	SDK initialization and global configuration	`initialize()`, `setSessionToken()`, `setRenderQuality()`, `deviceScore()`
`AvatarManager`	Avatar asset loading and caching	`load()`, `cancelLoad()`, `retrieve()`, `clear()`, `clearAll()`
`AvatarView`	3D rendering view	constructor, `dispose()`, `exportBitmap()`, `avatarTransform`, `getBoundingRect()`
`AvatarController`	Runtime communication and playback control	`start()`, `send()`, `yieldAudioData()`, `yieldFramesData()`, `pause()`, `resume()`, `interrupt()`

App ID is required on every integration path and identifies your Spatius application (it scopes which avatars you can load). Session Token is only required for Direct Mode (DrivingServiceMode.direct), where it authenticates the Motion Server WebSocket; RTC / Platform Integration / Backend Mode paths do not need it. See Credentials for the per-path breakdown.

For the minimum end-to-end integration walkthrough (install, build configuration, init, load, connect), see the Direct Mode guide for Web.

AvatarSDK

Main entry point for SDK initialization and global configuration.

import { AvatarSDK } from '@spatius/avatarkit'

Static properties

Property	Type	Description
`appId`	`string \| null`	Current App ID
`configuration`	`Configuration \| null`	Current SDK configuration
`sessionToken`	`string \| null`	Current Session Token
`userId`	`string \| null`	Current user ID (telemetry)
`version`	`string`	SDK version
`renderResolutionCapEnabled`	`boolean`	Whether render output resolution capping is enabled
`renderResolutionMaxHeight`	`number`	Maximum backing-buffer height used when capping is enabled

Static methods

initialize(appId, configuration)

Initialize the SDK. Must be called before any other operation.

await AvatarSDK.initialize('your-app-id', {
  region: 'us-west',
  drivingServiceMode: DrivingServiceMode.direct,    // optional, default: direct
  logLevel: LogLevel.warning,                       // optional, default: off
  renderQuality: RenderQuality.ultra,               // optional, default: ultra
  audioFormat: {                                    // optional
    channelCount: 1,
    sampleRate: 16000,
  },
})

setSessionToken(token)

Set the Session Token used to authenticate the Motion Server WebSocket. Only required in Direct Mode (DrivingServiceMode.direct); AvatarController.start() opens that WebSocket and authenticates with the token.

AvatarSDK.setSessionToken('your-session-token')

Required for DrivingServiceMode.direct before calling AvatarController.start().
Not required for DrivingServiceMode.backend (LiveKit Agents Integration, RTC Adapter path, Backend Mode). Those paths receive audio and motion data through their own transport and do not open a Motion Server WebSocket from the client; AvatarManager.load() fetches avatar metadata over an App-ID-scoped public endpoint.
The token must be obtained from your backend (see Session token API).
Maximum 24-hour validity.
Token must be paired with the App ID used in initialize().

setSessionToken() can be called before or after initialize(). If called before, the token is applied automatically during initialization.

setUserId(userId)

Set a user identifier for logging and telemetry.

AvatarSDK.setUserId('user-123')

setRenderQuality(quality)

Update the global rendering quality tier. The change takes effect on the next rendered frame across active AvatarView instances.

AvatarSDK.setRenderQuality(RenderQuality.high)

setRenderResolutionCap(enabled, maxHeight?)

Cap the render backing-buffer height while keeping the same CSS size. This is useful on high-DPI displays where rendering above the source avatar asset resolution adds cost without visible benefit.

AvatarSDK.setRenderResolutionCap(true, 1080)

deviceScore()

Run a short CPU/GPU benchmark and return device scores.

const { cpuScore, gpuScore } = await AvatarSDK.deviceScore()

isDeviceSupported()

Check whether the current device can run AvatarKit. In 1.2.0, this runs the same benchmark path used by deviceScore().

const supported = await AvatarSDK.isDeviceSupported()

cleanup()

Release all SDK resources. Call when the SDK is no longer needed.

AvatarSDK.cleanup()

AvatarManager

Handles avatar asset loading and caching. Access via the singleton AvatarManager.shared.

import { AvatarManager } from '@spatius/avatarkit'

Static properties

Property	Type	Description
`shared`	`AvatarManager`	Singleton instance

Instance methods

load(id, onProgress?, useCompressedModel?)

Load an avatar by ID. Downloads and caches the avatar’s assets.

const avatar = await AvatarManager.shared.load('avatar-id', (progress) => {
  switch (progress.type) {
    case 'downloading': {
      // progress.progress is in 0..1; multiply by 100 to render as a percentage.
      const percent = Math.round((progress.progress ?? 0) * 100)
      console.log(`Loading: ${percent}%`)
      break
    }
    case 'completed':
      console.log('Load complete')
      break
    case 'failed':
      console.error('Load failed:', progress.error)
      break
  }
})

Parameter	Type	Description
`id`	`string`	Avatar ID
`onProgress`	`(progress: LoadProgressInfo) => void`	Optional progress callback
`useCompressedModel`	`boolean`	Optional. Loads a smaller compressed model asset with minor quality tradeoff. Defaults to `false`.

Returns: Promise<Avatar>

cancelLoad(id)

Cancel a pending or running avatar load task.

const cancelled = AvatarManager.shared.cancelLoad('avatar-id')

retrieve(id)

Return a cached avatar instance, if available.

const cachedAvatar = AvatarManager.shared.retrieve('avatar-id')

clear(id)

Clear a specific avatar from cache.

AvatarManager.shared.clear('avatar-id')

clearAll()

Clear all cached avatar resources.

AvatarManager.shared.clearAll()

AvatarView

3D rendering view. Automatically creates a Canvas element and an associated AvatarController.

import { AvatarView } from '@spatius/avatarkit'

Constructor

const avatarView = new AvatarView(avatar, container)

Parameter	Type	Description
`avatar`	`Avatar`	Loaded avatar object
`container`	`HTMLElement`	Container element (canvas auto-fills the container)

Container requirement: the container element must have non-zero width and height. The canvas fills the container and auto-resizes via ResizeObserver.

Render over an Avatar Background

Download the optional 16:9 background from Spatius Studio and store it as an application asset. Set it on the same stage element that you pass to AvatarView; the SDK-created canvas has a transparent background.

<div id="avatar-stage"></div>

#avatar-stage {
  position: relative;
  width: 100%;
  aspect-ratio: 16 / 9;
  overflow: hidden;
  background: url('/backgrounds/my-avatar.webp') center / cover no-repeat;
}

const stage = document.querySelector<HTMLDivElement>('#avatar-stage')!
const avatar = await AvatarManager.shared.load('avatar-id')
const avatarView = new AvatarView(avatar, stage)

For square or portrait display windows, keep this inner stage at 16:9, center it, and clip it with an outer container. See Avatar Background for the shared cropping rules.

Instance properties

Property	Type	Description
`controller`	`AvatarController`	Communication controller (read-only)
`avatarTransform`	`{ x, y, scale }`	Avatar position and scale (see below)
`renderSize`	`{ width, height }`	Current canvas backing-buffer size in pixels
`onFirstRendering`	`() => void`	Callback fired when the first frame renders

Transform coordinates

Field	Range	Description
`x`	-1 to 1	Horizontal offset (-1 = left, 0 = center, 1 = right)
`y`	-1 to 1	Vertical offset (-1 = bottom, 0 = center, 1 = top)
`scale`	> 0	Scale factor (1.0 = original size)

Instance methods

dispose()

Release all view resources. Call when the view is no longer needed (see Lifecycle management for details).

avatarView.dispose()

exportBitmap()

Capture the current rendered avatar frame as a PNG Blob. Returns null if the canvas is not initialized or not currently rendering.

const png = await avatarView.exportBitmap()
if (png) {
  // Use the Blob for download, upload, or preview.
}

getCameraConfig() / updateCameraConfig(cameraConfig)

Read or update the camera used by the renderer.

const camera = avatarView.getCameraConfig()
if (camera) {
  avatarView.updateCameraConfig({
    ...camera,
    fov: 35,
  })
}

pauseRendering() / resumeRendering()

Pause or resume GPU/canvas rendering without stopping audio playback.

avatarView.pauseRendering()
avatarView.resumeRendering()

isRenderingEnabled()

Check whether the render loop is currently active.

const rendering = avatarView.isRenderingEnabled()

getBoundingRect()

Return the approximate avatar bounds in CSS pixels, or null if the view is not ready. Recalculate after container size or avatarTransform changes.

const rect = avatarView.getBoundingRect()
if (rect) {
  console.log(rect.x, rect.y, rect.width, rect.height)
}

AvatarController

Handles runtime communication with Motion Server and playback control.

// Accessed via AvatarView
const controller = avatarView.controller

Event callbacks

// Connection state changes
controller.onConnectionState = (state: ConnectionState) => {
  // 'disconnected' | 'connecting' | 'connected' | 'failed'
}

// Conversation state changes
controller.onConversationState = (state: ConversationState) => {
  // 'idle' | 'playing' | 'paused'
}

// Error events
controller.onError = (error: AvatarError) => {
  console.error('Error:', error.code, error.message)
}

// Animation type changes
controller.onAnimationState = (type: AnimationType) => {
  // 'idle' | 'mono'
}

// Only fires when frameStarvationMode is FrameStarvationMode.strictSync
controller.onPlaybackStall = (stalled: boolean) => {
  console.log('Playback stalled:', stalled)
}

Instance properties

Property	Type	Description
`frameStarvationMode`	`FrameStarvationMode`	Controls what happens when motion data cannot keep up with the audio clock. Defaults to `FrameStarvationMode.audioIndependent`.
`onFrameRateInfo`	`(info: FrameRateInfo) => void`	Optional callback for aggregated frame-rate metrics.
`frameRateMonitorEnabled`	`boolean`	Enables frame-rate monitoring. Defaults to `false`.

`DrivingServiceMode.direct` methods

Available when drivingServiceMode is DrivingServiceMode.direct (the Direct Mode path).

initializeAudioContext()

Initialize the audio context. Must be called inside a user-gesture handler (e.g. a click listener).

button.addEventListener('click', async () => {
  await controller.initializeAudioContext()
})

start()

Connect to Motion Server.

await controller.start()

send(audioData, end)

Send avatar speech audio. Returns the conversationId for the current round. For audio source and timing guidance, see Audio.

const conversationId = controller.send(
  audioData,  // PCM16, mono, ArrayBuffer
  end,        // true = end of conversation round
)

Parameter	Type	Description
`audioData`	`ArrayBuffer`	PCM16 mono audio data
`end`	`boolean`	Whether this is the final chunk of the current round

send() behavior:

end: false — continues the current conversation round.
end: true — marks the end of audio input for the current round. The avatar plays the remaining animation, then returns to idle (notified via onConversationState). Sending new audio after this starts a new round and interrupts any ongoing playback.

close()

Close the Motion Server connection.

controller.close()

`DrivingServiceMode.backend` methods

Available when drivingServiceMode is DrivingServiceMode.backend (the Backend Mode path).

yieldAudioData(audioData, end?)

Provide audio data when your backend owns the Motion Server connection.

const conversationId = controller.yieldAudioData(audioData, false)

Parameter	Type	Description
`audioData`	`Uint8Array`	Encoded audio payload from your backend transport
`end`	`boolean`	Optional. Whether this is the final audio payload for the current round.

Returns: string | null — conversation ID for this audio round, or null if audio playback could not start.

yieldFramesData(motionDataPayloads, conversationId)

Provide motion data payloads when using DrivingServiceMode.backend. The conversationId must match the one returned by the corresponding yieldAudioData() call.

const ended = controller.yieldFramesData(motionDataPayloads, conversationId)

Parameter	Type	Description
`motionDataPayloads`	`(Uint8Array \| ArrayBuffer)[]`	Motion data payloads from your backend transport
`conversationId`	`string`	Conversation ID returned by `yieldAudioData()`

Returns: boolean — true when the final motion data payload for that conversation has been received, otherwise false.

Common methods

Available in both DrivingServiceMode.direct and DrivingServiceMode.backend.

controller.pause()                     // Pause audio + animation
await controller.resume()              // Resume playback
controller.interrupt()                 // Stop current playback
controller.getAudioTime()              // Current audio playback time in seconds

// Volume control (avatar audio only, not system volume)
controller.setVolume(0.5)              // 0.0 to 1.0
controller.getVolume()                 // returns current volume

Types and Enums

Configuration

interface Configuration {
  region?: string                            // Defaults to 'us-west'
  drivingServiceMode?: DrivingServiceMode    // Default: DrivingServiceMode.direct
  logLevel?: LogLevel                        // Default: LogLevel.off
  audioFormat?: AudioFormat                  // Default: { channelCount: 1, sampleRate: 16000 }
  customEndpoint?: string                    // Advanced override
  renderQuality?: RenderQuality              // Default: RenderQuality.ultra
}

region selects the Spatius deployment region. It defaults to 'us-west'; supported values are 'us-west', 'ap-northeast', and 'cn-beijing'. See Regions for endpoint details and override options.

DrivingServiceMode

enum DrivingServiceMode {
  direct = 'direct',    // Client SDK handles the Motion Server connection (Direct Mode)
  backend = 'backend',  // Your backend handles the Motion Server connection (Backend Mode)
}

RenderQuality

enum RenderQuality {
  standard = 'standard',
  high = 'high',
  ultra = 'ultra',
}

FrameStarvationMode

enum FrameStarvationMode {
  audioIndependent = 'audioIndependent',
  strictSync = 'strictSync',
}

audioIndependent keeps audio playing while motion data catches up. strictSync pauses audio when motion data runs out and resumes when new motion data arrives; use onPlaybackStall to observe those transitions.

AnimationType

enum AnimationType {
  idle = 'idle',
  mono = 'mono',
}

LogLevel

enum LogLevel {
  off = 'off',          // No logging (default)
  error = 'error',      // Errors only
  warning = 'warning',  // Errors + warnings
  all = 'all',          // All logs
}

ConnectionState

Reported via onConnectionState. Only emitted when drivingServiceMode is DrivingServiceMode.direct.

enum ConnectionState {
  disconnected = 'disconnected',
  connecting = 'connecting',
  connected = 'connected',
  failed = 'failed',
}

ConversationState

Reported via onConversationState.

enum ConversationState {
  idle = 'idle',        // Breathing animation, waiting for input
  playing = 'playing',  // Active conversation playback
  paused = 'paused',    // Paused during playback
}

State transitions are notified immediately when the transition starts, not when the animation completes. For example, playing is reported as soon as the transition from idle begins.

LoadProgressInfo

type LoadProgressInfo = {
  type: 'downloading' | 'completed' | 'failed'
  progress?: number  // 0..1
  error?: Error
}

Multiply by 100 when rendering as a percentage in your UI.

AudioFormat

The SDK requires audio in mono PCM16 format.

interface AudioFormat {
  readonly channelCount: 1   // Fixed to mono
  readonly sampleRate: number // 8000 | 16000 | 22050 | 24000 | 32000 | 44100 | 48000
}

Property	Value
Format	PCM16 (16-bit signed integer, little-endian)
Channels	Mono (1 channel)
Sample rate	Configurable: 8000 / 16000 / 22050 / 24000 / 32000 / 44100 / 48000 Hz (default: 16000)
Data type	`ArrayBuffer` or `Uint8Array`

Data size: 1 second at 16 kHz = 16,000 samples × 2 bytes = 32,000 bytes.

Converting MP3 to PCM16

async function mp3ToPcm16(mp3File: File, targetSampleRate: number): Promise<ArrayBuffer> {
  const arrayBuffer = await mp3File.arrayBuffer()
  const audioContext = new AudioContext({ sampleRate: targetSampleRate })
  const audioBuffer = await audioContext.decodeAudioData(arrayBuffer.slice(0))

  const length = audioBuffer.length
  const channels = audioBuffer.numberOfChannels
  const pcm16Buffer = new ArrayBuffer(length * 2)
  const pcm16View = new DataView(pcm16Buffer)

  // Mix to mono if stereo
  const mono = channels === 1
    ? audioBuffer.getChannelData(0)
    : (() => {
        const mixed = new Float32Array(length)
        const left = audioBuffer.getChannelData(0)
        const right = audioBuffer.getChannelData(1)
        for (let i = 0; i < length; i++) mixed[i] = (left[i] + right[i]) / 2
        return mixed
      })()

  // Float32 → Int16
  for (let i = 0; i < length; i++) {
    const s = Math.max(-1, Math.min(1, mono[i]))
    pcm16View.setInt16(i * 2, s < 0 ? s * 0x8000 : s * 0x7FFF, true)
  }

  audioContext.close()
  return pcm16Buffer
}

Error Handling

AvatarError

import { AvatarError } from '@spatius/avatarkit'

try {
  await avatarView.controller.start()
} catch (error) {
  if (error instanceof AvatarError) {
    console.error('SDK error:', error.message, error.code)
  }
}

Error callback

avatarView.controller.onError = (error: AvatarError) => {
  console.error('Controller error:', error.code, error.message)
}

ErrorCode

AvatarError.code is one of the SDK string enum values below.

enum ErrorCode {
  appIDUnrecognized = 'appIDUnrecognized',
  sessionTokenInvalid = 'sessionTokenInvalid',
  sessionTokenExpired = 'sessionTokenExpired',
  insufficientBalance = 'insufficientBalance',
  concurrentLimitExceeded = 'concurrentLimitExceeded',
  avatarIDUnrecognized = 'avatarIDUnrecognized',
  failedToFetchAvatarMetadata = 'failedToFetchAvatarMetadata',
  invalidAvatarMetadata = 'invalidAvatarMetadata',
  failedToDownloadAvatarAssets = 'failedToDownloadAvatarAssets',
  websocketError = 'websocketError',
  websocketClosedAbnormally = 'websocketClosedAbnormally',
  websocketClosedUnexpected = 'websocketClosedUnexpected',
  sessionTimeout = 'sessionTimeout',
  connectionInProgress = 'connectionInProgress',
  networkLayerNotAvailable = 'networkLayerNotAvailable',
  playbackStartFailed = 'playbackStartFailed',
  playbackInitFailed = 'playbackInitFailed',
  audioOnlyInitFailed = 'audioOnlyInitFailed',
  noAudio = 'noAudio',
  audioContextNotInitialized = 'audioContextNotInitialized',
  animationPlayerNotInitialized = 'animationPlayerNotInitialized',
  serverError = 'serverError',
}

See Client Error Codes for recovery guidance and Server Error Codes for Console API and Motion Server errors.

Lifecycle Management

Avatar switching

// 1. Dispose the current view
currentAvatarView.dispose()

// 2. Load a new avatar
const newAvatar = await AvatarManager.shared.load('new-avatar-id')

// 3. Create a new view (reuse the same container)
currentAvatarView = new AvatarView(newAvatar, container)

// 4. Reconnect
await currentAvatarView.controller.initializeAudioContext()
await currentAvatarView.controller.start()

Resource cleanup

dispose() automatically cleans up:

WebSocket connections
Audio playback data and animation resources
Canvas elements and the render system
Event listeners and callbacks

Always call dispose() when the view is no longer needed. Failing to do so may cause memory leaks.

Fallback mechanism

If the WebSocket connection fails within 15 seconds, the SDK automatically enters audio-only fallback mode — audio continues playing without animation. This keeps playback uninterrupted when Motion Server is unreachable.

Fallback mode is interruptible like normal playback.
onConnectionState reports failed when the connection times out.

Browser Compatibility

Browser	Minimum version	Rendering
Chrome / Edge	90+	WebGPU (preferred)
Firefox	90+	WebGL
Safari	14+	WebGL
iOS Safari	14+	WebGL
Android Chrome	90+	WebGL

Common Issues

Issue	Cause	Solution
Audio not working	`initializeAudioContext()` not in a user gesture	Call it inside a `click` or `touchstart` handler.
Avatar not rendering	Container has zero dimensions	Set explicit `width` and `height` on the container.
WASM MIME type error	Build tool misconfigured	Use the Vite plugin or Next.js wrapper. See Toolchain Setup.
Session Token invalid	Token expired or not set	Refresh token from backend; call `setSessionToken()` before `start()`.
WebSocket connection failed	Network or auth issue	Check network connectivity and Session Token validity.

Complete Usage Example

import {
  AvatarSDK,
  AvatarManager,
  AvatarView,
  ConnectionState,
  ConversationState,
} from '@spatius/avatarkit'

class AvatarApp {
  private avatarView: AvatarView | null = null

  async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) {
    // Initialize the SDK
    await AvatarSDK.initialize(appId, { region: 'us-west' })
    AvatarSDK.setSessionToken(sessionToken)

    // Load the avatar
    const avatar = await AvatarManager.shared.load(avatarId)
    if (!avatar) throw new Error('Failed to load avatar')

    // Create the view
    this.avatarView = new AvatarView(avatar, container)
    this.avatarView.onFirstRendering = () => {
      console.log('First frame rendered')
    }

    // Wire up handlers
    this.avatarView.controller.onConnectionState = (state) => {
      console.log('Connection:', state)
    }
    this.avatarView.controller.onConversationState = (state) => {
      console.log('Conversation:', state)
    }
    this.avatarView.controller.onError = (error) => {
      console.error('Error:', error)
    }
  }

  // Must be called inside a user-gesture handler
  async start() {
    await this.avatarView?.controller.initializeAudioContext()
    await this.avatarView?.controller.start()
  }

  send(audioData: ArrayBuffer, isEnd: boolean) {
    this.avatarView?.controller.send(audioData, isEnd)
  }

  interrupt() {
    this.avatarView?.controller.interrupt()
  }

  dispose() {
    this.avatarView?.controller.close()
    this.avatarView?.dispose()
    this.avatarView = null
  }
}

​Quick Reference

​AvatarSDK

​Static properties

​Static methods

​initialize(appId, configuration)

​setSessionToken(token)

​setUserId(userId)

​setRenderQuality(quality)

​setRenderResolutionCap(enabled, maxHeight?)

​deviceScore()

​isDeviceSupported()

​cleanup()

​AvatarManager

​Static properties

​Instance methods

​load(id, onProgress?, useCompressedModel?)

​cancelLoad(id)

​retrieve(id)

​clear(id)

​clearAll()

​AvatarView

​Constructor

​Render over an Avatar Background

​Instance properties

​Instance methods

​dispose()

​exportBitmap()

​getCameraConfig() / updateCameraConfig(cameraConfig)

​pauseRendering() / resumeRendering()

​isRenderingEnabled()

​getBoundingRect()

​AvatarController

​Event callbacks

​Instance properties

​DrivingServiceMode.direct methods

​initializeAudioContext()

​start()

​send(audioData, end)

​close()

​DrivingServiceMode.backend methods

​yieldAudioData(audioData, end?)

​yieldFramesData(motionDataPayloads, conversationId)

​Common methods

​Types and Enums

​Configuration

​DrivingServiceMode

​RenderQuality

​FrameStarvationMode

​AnimationType

​LogLevel

​ConnectionState

​ConversationState

​LoadProgressInfo

​AudioFormat

​Error Handling

​AvatarError

​Error callback

​ErrorCode

​Lifecycle Management

​Avatar switching

​Resource cleanup

​Fallback mechanism

​Browser Compatibility

​Common Issues

​Complete Usage Example

Quick Reference

AvatarSDK

Static properties

Static methods

initialize(appId, configuration)

setSessionToken(token)

setUserId(userId)

setRenderQuality(quality)

setRenderResolutionCap(enabled, maxHeight?)

deviceScore()

isDeviceSupported()

cleanup()

AvatarManager

Static properties

Instance methods

load(id, onProgress?, useCompressedModel?)

cancelLoad(id)

retrieve(id)

clear(id)

clearAll()

AvatarView

Constructor

Render over an Avatar Background

Instance properties

Instance methods

dispose()

exportBitmap()

getCameraConfig() / updateCameraConfig(cameraConfig)

pauseRendering() / resumeRendering()

isRenderingEnabled()

getBoundingRect()

AvatarController

Event callbacks

Instance properties

`DrivingServiceMode.direct` methods

initializeAudioContext()

start()

send(audioData, end)

close()

`DrivingServiceMode.backend` methods

yieldAudioData(audioData, end?)

yieldFramesData(motionDataPayloads, conversationId)

Common methods

Types and Enums

Configuration

DrivingServiceMode

RenderQuality

FrameStarvationMode

AnimationType

LogLevel

ConnectionState

ConversationState

LoadProgressInfo

AudioFormat

Error Handling

AvatarError

Error callback

ErrorCode

Lifecycle Management

Avatar switching

Resource cleanup

Fallback mechanism

Browser Compatibility

Common Issues

Complete Usage Example