> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spatius.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# AvatarKit Web SDK Reference
> Complete API reference for @spatius/avatarkit (Web).
## Quick Reference
| Class | Purpose | Key methods |
| ------------------ | ------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `AvatarSDK` | SDK initialization and global configuration | `initialize()`, `setSessionToken()`, `setRenderQuality()`, `deviceScore()` |
| `AvatarManager` | Avatar asset loading and caching | `load()`, `cancelLoad()`, `retrieve()`, `clear()`, `clearAll()` |
| `AvatarView` | 3D rendering view | constructor, `dispose()`, `exportBitmap()`, `avatarTransform`, `getBoundingRect()` |
| `AvatarController` | Runtime communication and playback control | `start()`, `send()`, `yieldAudioData()`, `yieldFramesData()`, `pause()`, `resume()`, `interrupt()` |
**App ID** is required on every integration path and identifies your Spatius application (it scopes which avatars you can load). **Session Token** is only required for **Direct Mode** (`DrivingServiceMode.direct`), where it authenticates the Motion Server WebSocket; RTC / Platform Integration / Backend Mode paths do not need it. See [Credentials](/getting-started/credentials) for the per-path breakdown.
For the minimum end-to-end integration walkthrough (install, build configuration, init, load, connect), see the [Direct Mode guide for Web](/direct-mode/web).
***
## AvatarSDK
Main entry point for SDK initialization and global configuration.
```typescript theme={null}
import { AvatarSDK } from '@spatius/avatarkit'
```
### Static properties
| Property | Type | Description |
| ---------------------------- | ----------------------- | ---------------------------------------------------------- |
| `appId` | `string \| null` | Current App ID |
| `configuration` | `Configuration \| null` | Current SDK configuration |
| `sessionToken` | `string \| null` | Current Session Token |
| `userId` | `string \| null` | Current user ID (telemetry) |
| `version` | `string` | SDK version |
| `renderResolutionCapEnabled` | `boolean` | Whether render output resolution capping is enabled |
| `renderResolutionMaxHeight` | `number` | Maximum backing-buffer height used when capping is enabled |
### Static methods
#### initialize(appId, configuration)
Initialize the SDK. Must be called before any other operation.
```typescript theme={null}
await AvatarSDK.initialize('your-app-id', {
region: 'us-west',
drivingServiceMode: DrivingServiceMode.direct, // optional, default: direct
logLevel: LogLevel.warning, // optional, default: off
renderQuality: RenderQuality.ultra, // optional, default: ultra
audioFormat: { // optional
channelCount: 1,
sampleRate: 16000,
},
})
```
#### setSessionToken(token)
Set the Session Token used to authenticate the Motion Server WebSocket. **Only required in Direct Mode** (`DrivingServiceMode.direct`); `AvatarController.start()` opens that WebSocket and authenticates with the token.
```typescript theme={null}
AvatarSDK.setSessionToken('your-session-token')
```
* Required for `DrivingServiceMode.direct` before calling `AvatarController.start()`.
* **Not required** for `DrivingServiceMode.backend` (LiveKit Agents Integration, RTC Adapter path, Backend Mode). Those paths receive audio and motion data through their own transport and do not open a Motion Server WebSocket from the client; `AvatarManager.load()` fetches avatar metadata over an App-ID-scoped public endpoint.
* The token must be obtained from your backend (see [Session token API](/api-reference/api-reference)).
* Maximum 24-hour validity.
* Token must be paired with the App ID used in `initialize()`.
`setSessionToken()` can be called before or after `initialize()`. If called before, the token is applied automatically during initialization.
#### setUserId(userId)
Set a user identifier for logging and telemetry.
```typescript theme={null}
AvatarSDK.setUserId('user-123')
```
#### setRenderQuality(quality)
Update the global rendering quality tier. The change takes effect on the next rendered frame across active `AvatarView` instances.
```typescript theme={null}
AvatarSDK.setRenderQuality(RenderQuality.high)
```
#### setRenderResolutionCap(enabled, maxHeight?)
Cap the render backing-buffer height while keeping the same CSS size. This is useful on high-DPI displays where rendering above the source avatar asset resolution adds cost without visible benefit.
```typescript theme={null}
AvatarSDK.setRenderResolutionCap(true, 1080)
```
#### deviceScore()
Run a short CPU/GPU benchmark and return device scores.
```typescript theme={null}
const { cpuScore, gpuScore } = await AvatarSDK.deviceScore()
```
#### isDeviceSupported()
Check whether the current device can run AvatarKit. In `1.2.0`, this runs the same benchmark path used by `deviceScore()`.
```typescript theme={null}
const supported = await AvatarSDK.isDeviceSupported()
```
#### cleanup()
Release all SDK resources. Call when the SDK is no longer needed.
```typescript theme={null}
AvatarSDK.cleanup()
```
***
## AvatarManager
Handles avatar asset loading and caching. Access via the singleton `AvatarManager.shared`.
```typescript theme={null}
import { AvatarManager } from '@spatius/avatarkit'
```
### Static properties
| Property | Type | Description |
| -------- | --------------- | ------------------ |
| `shared` | `AvatarManager` | Singleton instance |
### Instance methods
#### load(id, onProgress?, useCompressedModel?)
Load an avatar by ID. Downloads and caches the avatar's assets.
```typescript theme={null}
const avatar = await AvatarManager.shared.load('avatar-id', (progress) => {
switch (progress.type) {
case 'downloading': {
// progress.progress is in 0..1; multiply by 100 to render as a percentage.
const percent = Math.round((progress.progress ?? 0) * 100)
console.log(`Loading: ${percent}%`)
break
}
case 'completed':
console.log('Load complete')
break
case 'failed':
console.error('Load failed:', progress.error)
break
}
})
```
| Parameter | Type | Description |
| -------------------- | -------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `id` | `string` | Avatar ID |
| `onProgress` | `(progress: LoadProgressInfo) => void` | Optional progress callback |
| `useCompressedModel` | `boolean` | Optional. Loads a smaller compressed model asset with minor quality tradeoff. Defaults to `false`. |
**Returns:** `Promise`
#### cancelLoad(id)
Cancel a pending or running avatar load task.
```typescript theme={null}
const cancelled = AvatarManager.shared.cancelLoad('avatar-id')
```
#### retrieve(id)
Return a cached avatar instance, if available.
```typescript theme={null}
const cachedAvatar = AvatarManager.shared.retrieve('avatar-id')
```
#### clear(id)
Clear a specific avatar from cache.
```typescript theme={null}
AvatarManager.shared.clear('avatar-id')
```
#### clearAll()
Clear all cached avatar resources.
```typescript theme={null}
AvatarManager.shared.clearAll()
```
***
## AvatarView
3D rendering view. Automatically creates a Canvas element and an associated `AvatarController`.
```typescript theme={null}
import { AvatarView } from '@spatius/avatarkit'
```
### Constructor
```typescript theme={null}
const avatarView = new AvatarView(avatar, container)
```
| Parameter | Type | Description |
| ----------- | ------------- | --------------------------------------------------- |
| `avatar` | `Avatar` | Loaded avatar object |
| `container` | `HTMLElement` | Container element (canvas auto-fills the container) |
**Container requirement:** the container element must have non-zero `width` and `height`. The canvas fills the container and auto-resizes via `ResizeObserver`.
### Instance properties
| Property | Type | Description |
| ------------------ | ------------------- | -------------------------------------------- |
| `controller` | `AvatarController` | Communication controller (read-only) |
| `avatarTransform` | `{ x, y, scale }` | Avatar position and scale (see below) |
| `renderSize` | `{ width, height }` | Current canvas backing-buffer size in pixels |
| `onFirstRendering` | `() => void` | Callback fired when the first frame renders |
**Transform coordinates**
| Field | Range | Description |
| ------- | ------- | ---------------------------------------------------- |
| `x` | -1 to 1 | Horizontal offset (-1 = left, 0 = center, 1 = right) |
| `y` | -1 to 1 | Vertical offset (-1 = bottom, 0 = center, 1 = top) |
| `scale` | > 0 | Scale factor (1.0 = original size) |
### Instance methods
#### dispose()
Release all view resources. Call when the view is no longer needed (see [Lifecycle management](#lifecycle-management) for details).
```typescript theme={null}
avatarView.dispose()
```
#### exportBitmap()
Capture the current rendered avatar frame as a PNG `Blob`. Returns `null` if the canvas is not initialized or not currently rendering.
```typescript theme={null}
const png = await avatarView.exportBitmap()
if (png) {
// Use the Blob for download, upload, or preview.
}
```
#### getCameraConfig() / updateCameraConfig(cameraConfig)
Read or update the camera used by the renderer.
```typescript theme={null}
const camera = avatarView.getCameraConfig()
if (camera) {
avatarView.updateCameraConfig({
...camera,
fov: 35,
})
}
```
#### pauseRendering() / resumeRendering()
Pause or resume GPU/canvas rendering without stopping audio playback.
```typescript theme={null}
avatarView.pauseRendering()
avatarView.resumeRendering()
```
#### isRenderingEnabled()
Check whether the render loop is currently active.
```typescript theme={null}
const rendering = avatarView.isRenderingEnabled()
```
#### getBoundingRect()
Return the approximate avatar bounds in CSS pixels, or `null` if the view is not ready. Recalculate after container size or `avatarTransform` changes.
```typescript theme={null}
const rect = avatarView.getBoundingRect()
if (rect) {
console.log(rect.x, rect.y, rect.width, rect.height)
}
```
***
## AvatarController
Handles runtime communication with Motion Server and playback control.
```typescript theme={null}
// Accessed via AvatarView
const controller = avatarView.controller
```
### Event callbacks
```typescript theme={null}
// Connection state changes
controller.onConnectionState = (state: ConnectionState) => {
// 'disconnected' | 'connecting' | 'connected' | 'failed'
}
// Conversation state changes
controller.onConversationState = (state: ConversationState) => {
// 'idle' | 'playing' | 'paused'
}
// Error events
controller.onError = (error: AvatarError) => {
console.error('Error:', error.code, error.message)
}
// Animation type changes
controller.onAnimationState = (type: AnimationType) => {
// 'idle' | 'mono'
}
// Only fires when frameStarvationMode is FrameStarvationMode.strictSync
controller.onPlaybackStall = (stalled: boolean) => {
console.log('Playback stalled:', stalled)
}
```
### Instance properties
| Property | Type | Description |
| ------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `frameStarvationMode` | `FrameStarvationMode` | Controls what happens when motion data cannot keep up with the audio clock. Defaults to `FrameStarvationMode.audioIndependent`. |
| `onFrameRateInfo` | `(info: FrameRateInfo) => void` | Optional callback for aggregated frame-rate metrics. |
| `frameRateMonitorEnabled` | `boolean` | Enables frame-rate monitoring. Defaults to `false`. |
### `DrivingServiceMode.direct` methods
Available when `drivingServiceMode` is `DrivingServiceMode.direct` (the Direct Mode path).
#### initializeAudioContext()
Initialize the audio context. **Must be called inside a user-gesture handler** (e.g. a `click` listener).
```typescript theme={null}
button.addEventListener('click', async () => {
await controller.initializeAudioContext()
})
```
#### start()
Connect to Motion Server.
```typescript theme={null}
await controller.start()
```
#### send(audioData, end)
Send avatar speech audio. Returns the `conversationId` for the current round.
For audio source and timing guidance, see [Audio](/concepts/audio).
```typescript theme={null}
const conversationId = controller.send(
audioData, // PCM16, mono, ArrayBuffer
end, // true = end of conversation round
)
```
| Parameter | Type | Description |
| ----------- | ------------- | ---------------------------------------------------- |
| `audioData` | `ArrayBuffer` | PCM16 mono audio data |
| `end` | `boolean` | Whether this is the final chunk of the current round |
**`send()` behavior:**
* `end: false` — continues the current conversation round.
* `end: true` — marks the end of audio input for the current round. The avatar plays the remaining animation, then returns to idle (notified via `onConversationState`). Sending new audio after this starts a new round and interrupts any ongoing playback.
#### close()
Close the Motion Server connection.
```typescript theme={null}
controller.close()
```
### `DrivingServiceMode.backend` methods
Available when `drivingServiceMode` is `DrivingServiceMode.backend` (the Backend Mode path).
#### yieldAudioData(audioData, end?)
Provide audio data when your backend owns the Motion Server connection.
```typescript theme={null}
const conversationId = controller.yieldAudioData(audioData, false)
```
| Parameter | Type | Description |
| ----------- | ------------ | ------------------------------------------------------------------------ |
| `audioData` | `Uint8Array` | Encoded audio payload from your backend transport |
| `end` | `boolean` | Optional. Whether this is the final audio payload for the current round. |
**Returns:** `string | null` — conversation ID for this audio round, or `null` if audio playback could not start.
#### yieldFramesData(motionDataPayloads, conversationId)
Provide motion data payloads when using `DrivingServiceMode.backend`. The `conversationId` must match the one returned by the corresponding `yieldAudioData()` call.
```typescript theme={null}
const ended = controller.yieldFramesData(motionDataPayloads, conversationId)
```
| Parameter | Type | Description |
| -------------------- | ------------------------------- | ------------------------------------------------ |
| `motionDataPayloads` | `(Uint8Array \| ArrayBuffer)[]` | Motion data payloads from your backend transport |
| `conversationId` | `string` | Conversation ID returned by `yieldAudioData()` |
**Returns:** `boolean` — `true` when the final motion data payload for that conversation has been received, otherwise `false`.
### Common methods
Available in both `DrivingServiceMode.direct` and `DrivingServiceMode.backend`.
```typescript theme={null}
controller.pause() // Pause audio + animation
await controller.resume() // Resume playback
controller.interrupt() // Stop current playback
controller.getAudioTime() // Current audio playback time in seconds
// Volume control (avatar audio only, not system volume)
controller.setVolume(0.5) // 0.0 to 1.0
controller.getVolume() // returns current volume
```
***
## Types and Enums
### Configuration
```typescript theme={null}
interface Configuration {
region?: string // Defaults to 'us-west'
drivingServiceMode?: DrivingServiceMode // Default: DrivingServiceMode.direct
logLevel?: LogLevel // Default: LogLevel.off
audioFormat?: AudioFormat // Default: { channelCount: 1, sampleRate: 16000 }
customEndpoint?: string // Advanced override
renderQuality?: RenderQuality // Default: RenderQuality.ultra
}
```
`region` selects the Spatius deployment region. It defaults to `'us-west'`; supported values are `'us-west'`, `'ap-northeast'`, and `'cn-beijing'`. See [Regions](/api-reference/regions) for endpoint details and override options.
### DrivingServiceMode
```typescript theme={null}
enum DrivingServiceMode {
direct = 'direct', // Client SDK handles the Motion Server connection (Direct Mode)
backend = 'backend', // Your backend handles the Motion Server connection (Backend Mode)
}
```
### RenderQuality
```typescript theme={null}
enum RenderQuality {
standard = 'standard',
high = 'high',
ultra = 'ultra',
}
```
### FrameStarvationMode
```typescript theme={null}
enum FrameStarvationMode {
audioIndependent = 'audioIndependent',
strictSync = 'strictSync',
}
```
`audioIndependent` keeps audio playing while motion data catches up. `strictSync` pauses audio when motion data runs out and resumes when new motion data arrives; use `onPlaybackStall` to observe those transitions.
### AnimationType
```typescript theme={null}
enum AnimationType {
idle = 'idle',
mono = 'mono',
}
```
### LogLevel
```typescript theme={null}
enum LogLevel {
off = 'off', // No logging (default)
error = 'error', // Errors only
warning = 'warning', // Errors + warnings
all = 'all', // All logs
}
```
### ConnectionState
Reported via `onConnectionState`. Only emitted when `drivingServiceMode` is `DrivingServiceMode.direct`.
```typescript theme={null}
enum ConnectionState {
disconnected = 'disconnected',
connecting = 'connecting',
connected = 'connected',
failed = 'failed',
}
```
### ConversationState
Reported via `onConversationState`.
```typescript theme={null}
enum ConversationState {
idle = 'idle', // Breathing animation, waiting for input
playing = 'playing', // Active conversation playback
paused = 'paused', // Paused during playback
}
```
State transitions are notified immediately when the transition starts, not when the animation completes. For example, `playing` is reported as soon as the transition from `idle` begins.
### LoadProgressInfo
```typescript theme={null}
type LoadProgressInfo = {
type: 'downloading' | 'completed' | 'failed'
progress?: number // 0..1
error?: Error
}
```
Multiply by 100 when rendering as a percentage in your UI.
### AudioFormat
The SDK requires audio in **mono PCM16** format.
```typescript theme={null}
interface AudioFormat {
readonly channelCount: 1 // Fixed to mono
readonly sampleRate: number // 8000 | 16000 | 22050 | 24000 | 32000 | 44100 | 48000
}
```
| Property | Value |
| --------------- | -------------------------------------------------------------------------------------- |
| **Format** | PCM16 (16-bit signed integer, little-endian) |
| **Channels** | Mono (1 channel) |
| **Sample rate** | Configurable: 8000 / 16000 / 22050 / 24000 / 32000 / 44100 / 48000 Hz (default: 16000) |
| **Data type** | `ArrayBuffer` or `Uint8Array` |
**Data size:** 1 second at 16 kHz = 16,000 samples × 2 bytes = 32,000 bytes.
```typescript theme={null}
async function mp3ToPcm16(mp3File: File, targetSampleRate: number): Promise {
const arrayBuffer = await mp3File.arrayBuffer()
const audioContext = new AudioContext({ sampleRate: targetSampleRate })
const audioBuffer = await audioContext.decodeAudioData(arrayBuffer.slice(0))
const length = audioBuffer.length
const channels = audioBuffer.numberOfChannels
const pcm16Buffer = new ArrayBuffer(length * 2)
const pcm16View = new DataView(pcm16Buffer)
// Mix to mono if stereo
const mono = channels === 1
? audioBuffer.getChannelData(0)
: (() => {
const mixed = new Float32Array(length)
const left = audioBuffer.getChannelData(0)
const right = audioBuffer.getChannelData(1)
for (let i = 0; i < length; i++) mixed[i] = (left[i] + right[i]) / 2
return mixed
})()
// Float32 → Int16
for (let i = 0; i < length; i++) {
const s = Math.max(-1, Math.min(1, mono[i]))
pcm16View.setInt16(i * 2, s < 0 ? s * 0x8000 : s * 0x7FFF, true)
}
audioContext.close()
return pcm16Buffer
}
```
***
## Error Handling
### AvatarError
```typescript theme={null}
import { AvatarError } from '@spatius/avatarkit'
try {
await avatarView.controller.start()
} catch (error) {
if (error instanceof AvatarError) {
console.error('SDK error:', error.message, error.code)
}
}
```
### Error callback
```typescript theme={null}
avatarView.controller.onError = (error: AvatarError) => {
console.error('Controller error:', error.code, error.message)
}
```
### ErrorCode
`AvatarError.code` is one of the SDK string enum values below.
```typescript theme={null}
enum ErrorCode {
appIDUnrecognized = 'appIDUnrecognized',
sessionTokenInvalid = 'sessionTokenInvalid',
sessionTokenExpired = 'sessionTokenExpired',
insufficientBalance = 'insufficientBalance',
concurrentLimitExceeded = 'concurrentLimitExceeded',
avatarIDUnrecognized = 'avatarIDUnrecognized',
failedToFetchAvatarMetadata = 'failedToFetchAvatarMetadata',
invalidAvatarMetadata = 'invalidAvatarMetadata',
failedToDownloadAvatarAssets = 'failedToDownloadAvatarAssets',
websocketError = 'websocketError',
websocketClosedAbnormally = 'websocketClosedAbnormally',
websocketClosedUnexpected = 'websocketClosedUnexpected',
sessionTimeout = 'sessionTimeout',
connectionInProgress = 'connectionInProgress',
networkLayerNotAvailable = 'networkLayerNotAvailable',
playbackStartFailed = 'playbackStartFailed',
playbackInitFailed = 'playbackInitFailed',
audioOnlyInitFailed = 'audioOnlyInitFailed',
noAudio = 'noAudio',
audioContextNotInitialized = 'audioContextNotInitialized',
animationPlayerNotInitialized = 'animationPlayerNotInitialized',
serverError = 'serverError',
}
```
See [Client Error Codes](/resources/client-error) for recovery guidance and [Server Error Codes](/resources/server-error) for Console API and Motion Server errors.
***
## Lifecycle Management
### Avatar switching
```typescript theme={null}
// 1. Dispose the current view
currentAvatarView.dispose()
// 2. Load a new avatar
const newAvatar = await AvatarManager.shared.load('new-avatar-id')
// 3. Create a new view (reuse the same container)
currentAvatarView = new AvatarView(newAvatar, container)
// 4. Reconnect
await currentAvatarView.controller.initializeAudioContext()
await currentAvatarView.controller.start()
```
### Resource cleanup
`dispose()` automatically cleans up:
* WebSocket connections
* Audio playback data and animation resources
* Canvas elements and the render system
* Event listeners and callbacks
Always call `dispose()` when the view is no longer needed. Failing to do so may cause memory leaks.
### Fallback mechanism
If the WebSocket connection fails within 15 seconds, the SDK automatically enters **audio-only fallback mode** — audio continues playing without animation. This keeps playback uninterrupted when Motion Server is unreachable.
* Fallback mode is interruptible like normal playback.
* `onConnectionState` reports `failed` when the connection times out.
***
## Browser Compatibility
| Browser | Minimum version | Rendering |
| -------------- | --------------- | ------------------ |
| Chrome / Edge | 90+ | WebGPU (preferred) |
| Firefox | 90+ | WebGL |
| Safari | 14+ | WebGL |
| iOS Safari | 14+ | WebGL |
| Android Chrome | 90+ | WebGL |
***
## Common Issues
| Issue | Cause | Solution |
| --------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------ |
| Audio not working | `initializeAudioContext()` not in a user gesture | Call it inside a `click` or `touchstart` handler. |
| Avatar not rendering | Container has zero dimensions | Set explicit `width` and `height` on the container. |
| WASM MIME type error | Build tool misconfigured | Use the Vite plugin or Next.js wrapper. See [Toolchain Setup](/sdk-reference/web-sdk/toolchain). |
| Session Token invalid | Token expired or not set | Refresh token from backend; call `setSessionToken()` before `start()`. |
| WebSocket connection failed | Network or auth issue | Check network connectivity and Session Token validity. |
***
## Complete Usage Example
```typescript theme={null}
import {
AvatarSDK,
AvatarManager,
AvatarView,
ConnectionState,
ConversationState,
} from '@spatius/avatarkit'
class AvatarApp {
private avatarView: AvatarView | null = null
async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) {
// Initialize the SDK
await AvatarSDK.initialize(appId, { region: 'us-west' })
AvatarSDK.setSessionToken(sessionToken)
// Load the avatar
const avatar = await AvatarManager.shared.load(avatarId)
if (!avatar) throw new Error('Failed to load avatar')
// Create the view
this.avatarView = new AvatarView(avatar, container)
this.avatarView.onFirstRendering = () => {
console.log('First frame rendered')
}
// Wire up handlers
this.avatarView.controller.onConnectionState = (state) => {
console.log('Connection:', state)
}
this.avatarView.controller.onConversationState = (state) => {
console.log('Conversation:', state)
}
this.avatarView.controller.onError = (error) => {
console.error('Error:', error)
}
}
// Must be called inside a user-gesture handler
async start() {
await this.avatarView?.controller.initializeAudioContext()
await this.avatarView?.controller.start()
}
send(audioData: ArrayBuffer, isEnd: boolean) {
this.avatarView?.controller.send(audioData, isEnd)
}
interrupt() {
this.avatarView?.controller.interrupt()
}
dispose() {
this.avatarView?.controller.close()
this.avatarView?.dispose()
this.avatarView = null
}
}
```