How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.
Avatar speech audio is the audio the Avatar should speak, usually TTS output from a voice-agent pipeline. It is not the user’s microphone audio.Motion Server uses this audio to generate synchronized motion data. AvatarKit plays the speech audio locally while rendering the resulting motion.
Send timing
Send each new audio chunk when it is generated. Do not delay chunks to match wall-clock playback time.Motion Server needs enough audio to generate the next motion window before AvatarKit consumes the current one. TTS output usually arrives faster than playback, allowing both buffers to stay ahead.
Do not feed audio that is already arriving at 1x playback speed back into Spatius chunk by chunk. The current segment can finish before the next synchronized segment is ready, causing playback to stall.
If your source is paced
If 1x playback-speed audio is your only source, pre-buffer it before forwarding it. Start each speech turn with enough buffered audio for Motion Server to remain ahead, then continue filling that buffer while sending. This adds startup latency but avoids repeated playback gaps.Reset the buffer when a turn is interrupted or a new turn begins.
Input format
The canonical input is mono 16-bit PCM (s16le) at the sample rate configured for the avatar view. Convert source audio first when its channels, encoding, or sample rate do not match.The audio format belongs to each avatar view, not to SDK initialization, so two views can run different sample rates. It can also be changed on a live view; a change interrupts the current turn and, in Direct Mode, disconnects, so reconnect before sending audio in the new format.Some SDK entry points also accept Opus. Format support and configuration differ by platform and integration, so use the exact SDK reference rather than mixing PCM and Opus within one session:Web · iOS · Android · Flutter
End input or interrupt
These actions have different meanings:
End input marks the final chunk of the current speech turn. Motion Server can finalize that turn, and AvatarKit continues playing the buffered result.
Interrupt cancels the active turn and clears queued audio and motion data. Use it for barge-in or when the response should stop immediately.
Do not use interruption as the normal end marker for every response.
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Spatius","colors":{"primary":"#9399E3","light":"#9A9AC0","dark":"#9A9AC0"},"logo":{"light":"https://mintcdn.com/spatius/wOjIDpP8OnEpRngp/images/spatius-logo-wordmark-black.png?fit=max\u0026auto=format\u0026n=wOjIDpP8OnEpRngp\u0026q=85\u0026s=3a9b7965a4bcfae37a65eefa457011ba","dark":"https://mintcdn.com/spatius/wOjIDpP8OnEpRngp/images/spatius-logo-wordmark-white.png?fit=max\u0026auto=format\u0026n=wOjIDpP8OnEpRngp\u0026q=85\u0026s=432ca82121812857d15cb7a85a90de0a"},"favicon":{"light":"https://mintcdn.com/spatius/KgLGkQWTJjnB2fTn/images/spatius-logo-mark-black.svg?fit=max\u0026auto=format\u0026n=KgLGkQWTJjnB2fTn\u0026q=85\u0026s=b3a8b579d89d1a5e93f73c732d20404d","dark":"https://mintcdn.com/spatius/KgLGkQWTJjnB2fTn/images/spatius-logo-mark-white.svg?fit=max\u0026auto=format\u0026n=KgLGkQWTJjnB2fTn\u0026q=85\u0026s=6f430113e3cf1c19f51a0a54c202046a"},"navbar":{"links":[{"icon":"discord","href":"https://discord.com/invite/9HGhZfHZh9","label":"Community"}],"primary":{"type":"button","label":"Studio","href":"https://app.spatius.ai"}},"integrations":{"posthog":{"apiKey":"phc_BJpW8hXVP84eK4bnQsRevfNy7uDdD7PAmxafqxLBerfj","sessionRecording":false}},"contextual":{"options":["copy","view","chatgpt","claude","perplexity","grok","mcp","cursor","vscode"],"display":"header"},"styling":{"latex":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"spatius","hasAudioNarration":false,"actualSubdomain":"spatius","customDomains":["docs.spatius.ai"],"gitSource":{"type":"github","owner":"spatius-ai","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":false,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6afd52a2dd0acd4b075a59f8","clientVersion":"0.0.4227","searchFilterCounts":[{"language":"default","tag":"Guides","count":24},{"language":"default","tag":"SDK Reference","count":12},{"language":"default","tag":"API Reference","count":12},{"language":"default","tag":"Integrations","count":9},{"language":"default","tag":"Client SDKs","count":6},{"language":"default","tag":"Start Here","count":5},{"language":"default","tag":"Concepts","count":5},{"language":"default","tag":"Avatar API","count":5},{"language":"default","tag":"Agora ConvoAI","count":4},{"language":"default","tag":"Video API","count":4}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"custom.css","content":".docs-page,:root{--spatius-selection-background: rgba(147, 153, 227, 0.22);--spatius-selection-text: #11131f}html.dark,.dark,[data-theme=\"dark\"]{--spatius-selection-background: rgba(173, 175, 244, 0.28);--spatius-selection-text: #ffffff}::selection{color:var(--spatius-selection-text);background:var(--spatius-selection-background)}.spatius-comparison-recommended{display:inline-block;margin-top:0.35rem;padding:0.12rem 0.48rem;border-radius:999px;color:#5559a9;background:rgba(147, 153, 227, 0.18);font-size:0.72rem;font-weight:650;line-height:1.35;white-space:nowrap}html.dark .spatius-comparison-recommended,.dark .spatius-comparison-recommended,[data-theme=\"dark\"] .spatius-comparison-recommended{color:#c3c5ff;background:rgba(173, 175, 244, 0.16)}.spatius-diagram.spatius-flow-explorer{--spatius-tour-surface: transparent;--spatius-tour-box: var(--spatius-diagram-node);--spatius-tour-border: var(--spatius-diagram-stroke);--spatius-tour-ink: var(--spatius-diagram-ink);--spatius-tour-muted: var(--spatius-diagram-muted);--spatius-tour-focus: var(--spatius-diagram-brand);--spatius-tour-node: var(--spatius-diagram-stroke);--spatius-tour-network-line: var(--spatius-diagram-muted);--spatius-tour-network-surface: rgba(56, 189, 248, 0.16);--spatius-tour-owned: var(--spatius-diagram-owned);--spatius-tour-managed: var(--spatius-diagram-managed);--spatius-tour-flow: var(--spatius-diagram-ink);--spatius-tour-audio: #2563eb;--spatius-tour-motion: #946200;container:spatius-flow / inline-size;margin:0;overflow:visible}html.dark .spatius-flow-explorer,.dark .spatius-flow-explorer,[data-theme=\"dark\"] .spatius-flow-explorer{--spatius-tour-audio: #60a5fa;--spatius-tour-motion: #facc15;--spatius-tour-network-surface: rgba(125, 211, 252, 0.16)}.spatius-flow-controls{display:grid;gap:0.2rem;padding:0.55rem 0.8rem 0}.spatius-flow-source-row{display:flex;align-items:flex-end;justify-content:space-between;min-width:0;gap:1rem}.spatius-flow-latency{display:inline-flex;flex:0 0 auto;align-items:center;gap:0.32rem;padding:0.5rem 0 0.62rem;color:var(--spatius-tour-ink);font-size:0.8rem;font-weight:700;line-height:1.25;white-space:nowrap}.spatius-flow-latency-title{margin-right:0.08rem}.spatius-flow-latency-term{display:inline-flex;align-items:center;gap:0.32rem}.spatius-flow-latency-stage{display:inline-flex;width:1.14rem;height:1.14rem;align-items:center;justify-content:center;border:1.25px solid var(--spatius-tour-ink);border-radius:50%;font-size:0.76rem;font-weight:700;line-height:1}.spatius-flow-latency-stage.is-variable{border-style:dashed}.spatius-flow-latency-plus{color:var(--spatius-tour-muted);font-weight:600}.spatius-flow-segmented{display:inline-flex;min-width:0;gap:0.25rem;padding:0;border:0;border-bottom:1px solid var(--spatius-tour-border);border-radius:0;background:transparent}.spatius-flow-segmented:not(.spatius-flow-mode-options){justify-self:start}.spatius-flow-segmented button{appearance:none;position:relative;min-width:5rem;padding:0.5rem 0.72rem 0.62rem;border:0;border-radius:0;color:var(--spatius-tour-muted);background:transparent;cursor:pointer;font:inherit;font-size:0.76rem;font-weight:600;line-height:1.25;white-space:nowrap;transition:color 140ms ease}.spatius-flow-segmented button::after{position:absolute;right:50%;bottom:-1px;left:50%;height:2px;border-radius:999px;background:var(--spatius-tour-focus);content:\"\";transition:right 160ms ease, left 160ms ease}.spatius-flow-segmented button:hover:not(:disabled){color:var(--spatius-tour-ink)}.spatius-flow-segmented button:disabled{color:var(--spatius-tour-muted);cursor:not-allowed;opacity:0.42}.spatius-flow-segmented button:focus-visible{outline:3px solid var(--spatius-tour-focus);outline-offset:2px}.spatius-flow-segmented button[data-selected=\"true\"]{color:var(--spatius-tour-ink);background:transparent;box-shadow:none}.spatius-flow-segmented button[data-selected=\"true\"]::after{right:0.72rem;left:0.72rem}.spatius-flow-mode-options{display:grid;width:100%;gap:0;grid-template-columns:repeat(3, minmax(0, 1fr))}.spatius-flow-mode-options button{min-width:0}.spatius-flow-diagram-shell{position:relative}.spatius-flow-diagram{overflow:hidden}.spatius-tour-stage{display:block;width:100%;min-width:0;height:auto;color:var(--spatius-tour-ink)}.spatius-tour-stage-surface{fill:var(--spatius-tour-surface)}.spatius-flow-explorer .spatius-tour-stage-surface,.spatius-flow-explorer .spatius-tour-network{filter:none}.spatius-tour-network{fill:var(--spatius-tour-network-surface)}.spatius-tour-network-edge{stroke:var(--spatius-tour-network-line);stroke-width:2;stroke-dasharray:6 7}.spatius-tour-zone-label,.spatius-tour-network-label{fill:var(--spatius-tour-ink);font-size:21px;font-weight:600}.spatius-tour-network-label{fill:var(--spatius-tour-muted);font-size:11px}.spatius-tour-network-icon circle,.spatius-tour-network-icon path{fill:none;stroke:var(--spatius-tour-network-line);stroke-width:1.5}.spatius-tour-node rect{fill:var(--spatius-tour-box);stroke:var(--spatius-tour-border);stroke-width:3;transition:stroke 180ms ease, filter 180ms ease}.spatius-tour-node.is-owner rect{fill:var(--spatius-tour-owned)}.spatius-tour-node.is-service rect{fill:var(--spatius-tour-managed);stroke:var(--spatius-diagram-brand)}.spatius-tour-node.is-sdk rect{fill:var(--spatius-tour-box);stroke:var(--spatius-tour-border)}.spatius-tour-node.is-room rect{stroke:var(--spatius-tour-node);stroke-width:3}.spatius-tour-node.is-lit rect{animation:spatius-tour-node-pulse 260ms ease-out}.spatius-tour-node-label,.spatius-tour-owner-label{fill:var(--spatius-tour-ink);font-size:17px;font-weight:600}.spatius-tour-owner-label{font-size:15px}.spatius-tour-wire{fill:none;stroke:var(--spatius-tour-flow);stroke-width:2.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none}.spatius-tour-wire.is-crossing{stroke-dasharray:7 8}.spatius-tour-footer-edge{stroke:var(--spatius-tour-border);stroke-width:1.5}.spatius-tour-legend-label{fill:var(--spatius-tour-ink);font-size:16px;font-weight:600}.spatius-tour-legend-orb.is-audio{color:var(--spatius-tour-audio)}.spatius-tour-legend-orb.is-motion{color:var(--spatius-tour-motion)}.spatius-tour-legend-orb-halo{fill:currentColor;opacity:0.3}.spatius-tour-legend-orb-core{fill:currentColor}.spatius-tour-arrow{fill:var(--spatius-tour-flow)}.spatius-tour-wire.is-active{animation:spatius-tour-wire-travel 248ms linear infinite}.spatius-tour-wire.is-active.is-crossing{animation-duration:1984ms}.spatius-tour-stage-number circle{fill:var(--spatius-tour-box);stroke:var(--spatius-tour-border);stroke-width:2}.spatius-tour-stage-number.is-variable circle{stroke-dasharray:4 3}.spatius-tour-stage-number text{fill:var(--spatius-tour-ink);font-size:12px;font-weight:700}.spatius-flow-status,.spatius-flow-seo-copy{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0, 0, 0, 0);clip-path:inset(50%);white-space:nowrap;border:0}.spatius-tour-packet{pointer-events:none}.spatius-tour-packet.is-audio{color:var(--spatius-tour-audio)}.spatius-tour-packet.is-motion{color:var(--spatius-tour-motion)}.spatius-tour-packet-core{fill:#ffffff}.spatius-tour-packet-halo{fill:currentColor;opacity:0.72}.spatius-tour-packet-glow{fill:currentColor;opacity:0.42;filter:url(#spatius-tour-packet-glow)}@keyframes spatius-tour-wire-travel{to{stroke-dashoffset:-30}}@keyframes spatius-tour-node-pulse{0%{filter:drop-shadow(0 0 0 transparent)}42%{filter:drop-shadow(0 0 12px color-mix(in srgb, var(--spatius-tour-node) 70%, transparent))}100%{filter:drop-shadow(0 0 0 transparent)}}@media (max-width: 720px){.spatius-flow-mode-options{grid-template-columns:minmax(0, 1fr)}}@media (max-width: 460px){.spatius-flow-controls{padding:0.65rem}.spatius-flow-source-row{align-items:stretch;flex-direction:column;gap:0}.spatius-flow-latency{align-self:flex-end}}@media (prefers-reduced-motion: reduce){.spatius-flow-segmented button,.spatius-flow-segmented button::after{transition:none}.spatius-tour-wire.is-active,.spatius-tour-node.is-lit rect{animation:none}.spatius-tour-packets{display:none}}.step-container{margin-bottom:0.25rem!important}.step-container+.step-container{margin-top:0.25rem!important}[class*=\"steps\"]{margin-top:1.5rem!important}[class*=\"steps\"] .step-container{margin-bottom:0.25rem!important}[class*=\"steps\"] .accordion-item{margin-bottom:0.25rem!important}.spatius-diagram{--spatius-diagram-ink: #171821;--spatius-diagram-box: rgba(248, 249, 255, 0.94);--spatius-diagram-managed: rgba(149, 153, 227, 0.1);--spatius-diagram-stroke: rgba(52, 55, 76, 0.38);--spatius-diagram-brand: #7478d9;--spatius-diagram-muted: rgba(60, 63, 82, 0.58);--spatius-diagram-blue: #7478e7;--spatius-diagram-livekit: rgb(31, 213, 249);--spatius-diagram-livekit-surface: rgba(31, 213, 249, 0.12);--spatius-diagram-red: #5fae88;--spatius-diagram-green: rgba(54, 105, 82, 0.82);--spatius-diagram-owned: rgba(54, 105, 82, 0.1);--spatius-diagram-node: rgba(246, 247, 253, 0.98);--spatius-diagram-white: #191a24;--spatius-diagram-muted-dark: rgba(64, 67, 86, 0.52);--spatius-diagram-glow: rgba(116, 120, 231, 0.28);--spatius-diagram-edge: rgba(255, 255, 255, 0.72);--spatius-diagram-warning: #d65f66;--spatius-diagram-warning-surface: rgba(214, 95, 102, 0.1);--spatius-diagram-warning-stroke: rgba(214, 95, 102, 0.48);width:100%;margin:0;overflow:visible;border-radius:6px;color:var(--spatius-diagram-ink)}.spatius-agora-diagram{--spatius-diagram-agora: rgb(91, 193, 251);--spatius-diagram-agora-surface: rgba(91, 193, 251, 0.12);--spatius-diagram-agora-stroke: rgba(91, 193, 251, 0.54)}.spatius-diagram svg{display:block;width:100%;max-width:100%;height:auto;color:var(--spatius-diagram-ink)}.spatius-diagram rect:not([fill=\"none\"]){filter:drop-shadow(0 18px 26px rgba(20, 22, 36, 0.1))}.spatius-diagram text{font-family:inherit;letter-spacing:0}.spatius-diagram-zone{fill:var(--spatius-diagram-managed);stroke-width:2.5}.spatius-diagram-orbit{fill:none;stroke:var(--spatius-diagram-brand);stroke-dasharray:2 10;stroke-linecap:round;stroke-width:2;opacity:0.55}.spatius-audio-bar{animation:spatius-audio-bar 1.08s ease-in-out infinite;fill:var(--spatius-diagram-brand);filter:drop-shadow(0 0 7px var(--spatius-diagram-glow));transform-box:fill-box;transform-origin:center}.spatius-audio-bar:nth-child(2){animation-delay:-0.86s}.spatius-audio-bar:nth-child(3){animation-delay:-0.62s}.spatius-audio-bar:nth-child(4){animation-delay:-0.44s}.spatius-audio-bar:nth-child(5){animation-delay:-0.18s}.spatius-audio-visualizer{opacity:0.96}.spatius-agent-pipeline,.spatius-agent-speech{pointer-events:none}.spatius-agent-pipeline{animation:spatius-agent-pipeline 9s ease-in-out infinite}.spatius-agent-speech{animation:spatius-agent-speech 9s ease-in-out infinite;opacity:0}.spatius-diagram-flow{filter:drop-shadow(0 0 5px var(--spatius-diagram-glow));stroke-dasharray:12 12;animation:spatius-diagram-flow 1.6s linear infinite}.spatius-audio-timing-diagram{--spatius-timing-good-distance: 108px;--spatius-timing-bad-distance: 108px;--spatius-diagram-warning: #c85b63;--spatius-diagram-warning-surface: rgba(200, 91, 99, 0.08);--spatius-diagram-warning-stroke: rgba(200, 91, 99, 0.42)}.spatius-audio-timing-diagram rect:not([fill=\"none\"]){filter:drop-shadow(0 12px 18px rgba(20, 22, 36, 0.08))}.spatius-timing-packet{filter:drop-shadow(0 0 8px var(--spatius-diagram-glow));transform-box:fill-box;transform-origin:center}.spatius-timing-packet-good{animation:spatius-timing-packet-good 1.45s ease-in-out infinite}.spatius-timing-packet-bad{animation:spatius-timing-packet-bad 4.2s ease-in-out infinite}.spatius-timing-delay-1{animation-delay:-0.48s}.spatius-timing-delay-2{animation-delay:-0.96s}.spatius-timing-window-pulse rect{animation:spatius-timing-window-pulse 2.2s ease-in-out infinite;transform-box:fill-box;transform-origin:center}.spatius-timing-window-pulse rect:nth-child(2){animation-delay:-1.45s}.spatius-timing-window-pulse rect:nth-child(3){animation-delay:-0.72s}.spatius-timing-blocker{animation:spatius-timing-blocker 2s ease-in-out infinite;filter:drop-shadow(0 0 6px rgba(200, 91, 99, 0.22));transform-box:fill-box;transform-origin:center}.spatius-timing-flow-bad{animation-duration:3.2s;filter:drop-shadow(0 0 4px rgba(200, 91, 99, 0.18))}@keyframes spatius-audio-bar{0%,100%{transform:scaleY(0.42)}45%{transform:scaleY(1)}70%{transform:scaleY(0.64)}}@keyframes spatius-diagram-flow{to{stroke-dashoffset:-24}}@keyframes spatius-timing-packet-good{0%{opacity:0;transform:translateX(0) scale(0.82)}15%,74%{opacity:1}100%{opacity:0;transform:translateX(var(--spatius-timing-good-distance)) scale(1)}}@keyframes spatius-timing-packet-bad{0%{opacity:0;transform:translateX(0) scale(0.82)}24%,70%{opacity:1}100%{opacity:0;transform:translateX(var(--spatius-timing-bad-distance)) scale(1)}}@keyframes spatius-timing-window-pulse{0%,100%{opacity:0.52;transform:scale(0.96)}45%{opacity:1;transform:scale(1.04)}}@keyframes spatius-timing-blocker{0%,100%{opacity:0.78;transform:scale(0.96)}50%{opacity:1;transform:scale(1.04)}}@keyframes spatius-agent-pipeline{0%,42%{opacity:1}48%,92%{opacity:0}98%,100%{opacity:1}}@keyframes spatius-agent-speech{0%,42%{opacity:0}48%,92%{opacity:1}98%,100%{opacity:0}}@media (prefers-reduced-motion: reduce){.spatius-diagram-flow{animation:none;stroke-dasharray:none}.spatius-audio-bar{animation:none}.spatius-agent-pipeline{animation:none;opacity:1}.spatius-agent-speech{animation:none;opacity:0}.spatius-timing-packet,.spatius-timing-window-pulse rect,.spatius-timing-blocker{animation:none}}html.dark .spatius-diagram,.dark .spatius-diagram,[data-theme=\"dark\"] .spatius-diagram{--spatius-diagram-ink: #f7f7fb;--spatius-diagram-box: rgba(31, 34, 48, 0.88);--spatius-diagram-managed: rgba(147, 153, 227, 0.16);--spatius-diagram-stroke: rgba(234, 235, 255, 0.24);--spatius-diagram-brand: #adaff4;--spatius-diagram-muted: rgba(232, 233, 246, 0.6);--spatius-diagram-blue: #9399e3;--spatius-diagram-livekit: rgb(31, 213, 249);--spatius-diagram-livekit-surface: rgba(31, 213, 249, 0.16);--spatius-diagram-red: #7bd4a8;--spatius-diagram-green: rgba(75, 126, 97, 0.78);--spatius-diagram-owned: rgba(75, 126, 97, 0.18);--spatius-diagram-node: rgba(43, 48, 67, 0.92);--spatius-diagram-white: #f7f7fb;--spatius-diagram-muted-dark: rgba(230, 230, 240, 0.5);--spatius-diagram-glow: rgba(147, 153, 227, 0.46);--spatius-diagram-edge: rgba(255, 255, 255, 0.2);--spatius-diagram-warning: #ff8a92;--spatius-diagram-warning-surface: rgba(255, 138, 146, 0.16);--spatius-diagram-warning-stroke: rgba(255, 138, 146, 0.58)}html.dark .spatius-agora-diagram,.dark .spatius-agora-diagram,[data-theme=\"dark\"] .spatius-agora-diagram{--spatius-diagram-agora-surface: rgba(91, 193, 251, 0.16);--spatius-diagram-agora-stroke: rgba(91, 193, 251, 0.62)}html.dark .spatius-audio-timing-diagram,.dark .spatius-audio-timing-diagram,[data-theme=\"dark\"] .spatius-audio-timing-diagram{--spatius-diagram-warning: #f27f88;--spatius-diagram-warning-surface: rgba(242, 127, 136, 0.13);--spatius-diagram-warning-stroke: rgba(242, 127, 136, 0.52)}.walkthrough-code pre{max-height:460px;overflow:auto;-webkit-overflow-scrolling:touch}.walkthrough-code [data-content-type=\"codeBlock\"]{max-height:460px;overflow:auto;-webkit-overflow-scrolling:touch}"}]}
{"page":{"pagePath":"concepts/audio","slug":"/concepts/audio","pageMetadata":{"title":"Audio","sidebarTitle":"Audio","description":"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.","href":"/concepts/audio"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];const AudioSendTimingDiagram=()=>{return _jsx(`div`,{className:`spatius-diagram spatius-audio-timing-diagram not-prose`,\"aria-label\":`Comparison of generated audio and paced audio for client playback buffering`,children:_jsxs(`svg`,{viewBox:`0 0 980 980`,role:`img`,children:[_jsxs(`defs`,{children:[_jsx(`marker`,{id:`spatius-audio-timing-good`,viewBox:`0 0 12 10`,refX:`10.5`,refY:`5`,markerWidth:`4.5`,markerHeight:`6`,orient:`auto-start-reverse`,children:_jsx(`path`,{d:`M0 0L12 5L0 10Z`,fill:`var(--spatius-diagram-ink)`})}),_jsx(`marker`,{id:`spatius-audio-timing-bad`,viewBox:`0 0 12 10`,refX:`10.5`,refY:`5`,markerWidth:`4.5`,markerHeight:`6`,orient:`auto-start-reverse`,children:_jsx(`path`,{d:`M0 0L12 5L0 10Z`,fill:`var(--spatius-diagram-warning)`})})]}),_jsx(`text`,{x:`305`,y:`54`,textAnchor:`middle`,fill:`var(--spatius-diagram-red)`,fontSize:`24`,fontWeight:`600`,children:`Generated Audio Path`}),_jsx(`text`,{x:`770`,y:`54`,textAnchor:`middle`,fill:`var(--spatius-diagram-brand)`,fontSize:`24`,fontWeight:`600`,children:`Inference + Playback`}),_jsx(`rect`,{x:`40`,y:`100`,width:`530`,height:`340`,rx:`18`,fill:`var(--spatius-diagram-owned)`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`305`,y:`136`,textAnchor:`middle`,fill:`var(--spatius-diagram-red)`,fontSize:`18`,fontWeight:`600`,children:`Works: send new TTS chunks immediately`}),_jsx(`rect`,{x:`80`,y:`180`,width:`170`,height:`102`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`165`,y:`222`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`28`,fontWeight:`500`,children:`TTS`}),_jsx(`text`,{x:`165`,y:`254`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`18`,fontWeight:`500`,children:`new chunks`}),_jsx(`rect`,{x:`360`,y:`180`,width:`170`,height:`102`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`445`,y:`220`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`25`,fontWeight:`500`,children:`SDK send`}),_jsx(`text`,{x:`445`,y:`252`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`18`,fontWeight:`500`,children:`immediate`}),_jsx(`polyline`,{points:`262,231 348,231`,className:`spatius-diagram-flow`,fill:`none`,stroke:`var(--spatius-diagram-ink)`,strokeWidth:`4`,strokeLinecap:`round`,markerEnd:`url(#spatius-audio-timing-good)`}),_jsx(`text`,{x:`85`,y:`334`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`600`,children:`generation time`}),_jsx(`line`,{x1:`85`,y1:`360`,x2:`525`,y2:`360`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`4`,strokeLinecap:`round`}),_jsxs(`g`,{className:`spatius-timing-fast-chunks`,children:[_jsx(`rect`,{x:`255`,y:`336`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-managed)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`279`,y:`367`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`1`}),_jsx(`rect`,{x:`318`,y:`336`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-managed)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`342`,y:`367`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`2`}),_jsx(`rect`,{x:`381`,y:`336`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-managed)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`405`,y:`367`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`3`})]}),_jsx(`text`,{x:`305`,y:`416`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`500`,children:`chunks arrive ahead of playback`}),_jsx(`rect`,{x:`620`,y:`100`,width:`300`,height:`340`,rx:`18`,fill:`var(--spatius-diagram-managed)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`3`}),_jsx(`rect`,{x:`670`,y:`158`,width:`200`,height:`120`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`3`}),_jsxs(`text`,{x:`770`,y:`203`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`28`,fontWeight:`500`,children:[_jsx(`tspan`,{x:`770`,children:`Motion`}),_jsx(`tspan`,{x:`770`,dy:`36`,children:`Server`})]}),_jsxs(`g`,{className:`spatius-timing-window-pulse`,children:[_jsx(`rect`,{x:`682`,y:`318`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`}),_jsx(`rect`,{x:`744`,y:`318`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`}),_jsx(`rect`,{x:`806`,y:`318`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-brand)`,strokeWidth:`2.5`})]}),_jsx(`text`,{x:`770`,y:`396`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`500`,children:`buffer stays ahead`}),_jsx(`polyline`,{points:`540,231 658,231`,className:`spatius-diagram-flow`,fill:`none`,stroke:`var(--spatius-diagram-ink)`,strokeWidth:`4`,strokeLinecap:`round`,markerEnd:`url(#spatius-audio-timing-good)`}),_jsx(`circle`,{className:`spatius-timing-packet spatius-timing-packet-good`,cx:`550`,cy:`231`,r:`9`,fill:`var(--spatius-diagram-brand)`}),_jsx(`circle`,{className:`spatius-timing-packet spatius-timing-packet-good spatius-timing-delay-1`,cx:`550`,cy:`231`,r:`9`,fill:`var(--spatius-diagram-brand)`}),_jsx(`circle`,{className:`spatius-timing-packet spatius-timing-packet-good spatius-timing-delay-2`,cx:`550`,cy:`231`,r:`9`,fill:`var(--spatius-diagram-brand)`}),_jsx(`text`,{x:`600`,y:`207`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`600`,children:`fast`}),_jsx(`text`,{x:`305`,y:`526`,textAnchor:`middle`,fill:`var(--spatius-diagram-warning)`,fontSize:`24`,fontWeight:`600`,children:`Paced Audio Path`}),_jsx(`text`,{x:`770`,y:`526`,textAnchor:`middle`,fill:`var(--spatius-diagram-warning)`,fontSize:`24`,fontWeight:`600`,children:`Playback Buffer`}),_jsx(`rect`,{x:`40`,y:`570`,width:`530`,height:`340`,rx:`18`,fill:`var(--spatius-diagram-warning-surface)`,stroke:`var(--spatius-diagram-warning-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`305`,y:`606`,textAnchor:`middle`,fill:`var(--spatius-diagram-warning)`,fontSize:`18`,fontWeight:`600`,children:`Avoid: send 1x playback-speed output`}),_jsx(`rect`,{x:`80`,y:`650`,width:`170`,height:`102`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`165`,y:`690`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`24`,fontWeight:`500`,children:`Paced audio`}),_jsx(`text`,{x:`165`,y:`722`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`18`,fontWeight:`500`,children:`already heard`}),_jsx(`rect`,{x:`360`,y:`650`,width:`170`,height:`102`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`3`}),_jsx(`text`,{x:`445`,y:`690`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`25`,fontWeight:`500`,children:`Decode`}),_jsx(`text`,{x:`445`,y:`722`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`18`,fontWeight:`500`,children:`paced at 1x`}),_jsx(`polyline`,{points:`262,701 348,701`,className:`spatius-diagram-flow spatius-timing-flow-bad`,fill:`none`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`4`,strokeLinecap:`round`,markerEnd:`url(#spatius-audio-timing-bad)`}),_jsx(`text`,{x:`85`,y:`804`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`600`,children:`playback time`}),_jsx(`line`,{x1:`85`,y1:`830`,x2:`525`,y2:`830`,stroke:`var(--spatius-diagram-stroke)`,strokeWidth:`4`,strokeLinecap:`round`}),_jsxs(`g`,{className:`spatius-timing-slow-chunks`,children:[_jsx(`rect`,{x:`225`,y:`806`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-warning-surface)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`249`,y:`837`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`1`}),_jsx(`rect`,{x:`347`,y:`806`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-warning-surface)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`371`,y:`837`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`2`}),_jsx(`rect`,{x:`469`,y:`806`,width:`48`,height:`48`,rx:`8`,fill:`var(--spatius-diagram-warning-surface)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`}),_jsx(`text`,{x:`493`,y:`837`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`18`,fontWeight:`700`,children:`3`})]}),_jsx(`text`,{x:`305`,y:`886`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`500`,children:`chunks arrive only when listeners hear them`}),_jsx(`rect`,{x:`620`,y:`570`,width:`300`,height:`340`,rx:`18`,fill:`var(--spatius-diagram-warning-surface)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`3`}),_jsx(`rect`,{x:`670`,y:`625`,width:`200`,height:`118`,rx:`12`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`3`}),_jsxs(`text`,{x:`770`,y:`670`,textAnchor:`middle`,fill:`var(--spatius-diagram-ink)`,fontSize:`28`,fontWeight:`500`,children:[_jsx(`tspan`,{x:`770`,children:`Motion`}),_jsx(`tspan`,{x:`770`,dy:`36`,children:`Server`})]}),_jsx(`text`,{x:`770`,y:`772`,textAnchor:`middle`,fill:`var(--spatius-diagram-warning)`,fontSize:`20`,fontWeight:`700`,children:`motion returned`}),_jsx(`text`,{x:`770`,y:`884`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`18`,fontWeight:`500`,children:`playback can stall`}),_jsx(`polyline`,{points:`540,701 658,701`,className:`spatius-diagram-flow spatius-timing-flow-bad`,fill:`none`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`4`,strokeLinecap:`round`,markerEnd:`url(#spatius-audio-timing-bad)`}),_jsx(`circle`,{className:`spatius-timing-packet spatius-timing-packet-bad`,cx:`550`,cy:`701`,r:`9`,fill:`var(--spatius-diagram-warning)`}),_jsx(`text`,{x:`600`,y:`677`,textAnchor:`middle`,fill:`var(--spatius-diagram-muted)`,fontSize:`17`,fontWeight:`600`,children:`1x`}),_jsxs(`g`,{className:`spatius-timing-window-pulse`,children:[_jsx(`rect`,{x:`682`,y:`802`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`}),_jsx(`rect`,{x:`744`,y:`802`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`}),_jsx(`rect`,{x:`806`,y:`802`,width:`52`,height:`46`,rx:`8`,fill:`var(--spatius-diagram-node)`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`2.5`})]}),_jsxs(`g`,{className:`spatius-timing-blocker`,children:[_jsx(`line`,{x1:`742`,y1:`795`,x2:`798`,y2:`855`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`8`,strokeLinecap:`round`}),_jsx(`line`,{x1:`798`,y1:`795`,x2:`742`,y2:`855`,stroke:`var(--spatius-diagram-warning)`,strokeWidth:`8`,strokeLinecap:`round`})]})]})})};function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,p:`p`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{Frame,Heading,Warning,_MdxComponentBoundary}=_components;if(!Frame)_missingMdxReference(`Frame`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Warning)_missingMdxReference(`Warning`,true);if(!_MdxComponentBoundary)_missingMdxReference(`_MdxComponentBoundary`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`Avatar speech audio`}),` is the audio the Avatar should speak, usually TTS output from a voice-agent pipeline. It is not the user’s microphone audio.`]}),`\n`,_jsx(_components.p,{children:`Motion Server uses this audio to generate synchronized motion data. AvatarKit plays the speech audio locally while rendering the resulting motion.`}),`\n`,_jsx(Heading,{level:`2`,id:`send-timing`,children:`Send timing`}),`\n`,_jsx(_components.p,{children:`Send each new audio chunk when it is generated. Do not delay chunks to match wall-clock playback time.`}),`\n`,_jsx(_components.p,{children:`Motion Server needs enough audio to generate the next motion window before AvatarKit consumes the current one. TTS output usually arrives faster than playback, allowing both buffers to stay ahead.`}),`\n`,_jsx(Frame,{children:_jsx(_MdxComponentBoundary,{name:`AudioSendTimingDiagram`,children:_jsx(AudioSendTimingDiagram,{})})}),`\n`,_jsx(Warning,{children:_jsx(_components.p,{children:`Do not feed audio that is already arriving at 1x playback speed back into Spatius chunk by chunk. The current segment can finish before the next synchronized segment is ready, causing playback to stall.`})}),`\n`,_jsx(Heading,{level:`3`,id:`if-your-source-is-paced`,children:`If your source is paced`}),`\n`,_jsx(_components.p,{children:`If 1x playback-speed audio is your only source, pre-buffer it before forwarding it. Start each speech turn with enough buffered audio for Motion Server to remain ahead, then continue filling that buffer while sending. This adds startup latency but avoids repeated playback gaps.`}),`\n`,_jsx(_components.p,{children:`Reset the buffer when a turn is interrupted or a new turn begins.`}),`\n`,_jsx(Heading,{level:`2`,id:`input-format`,children:`Input format`}),`\n`,_jsxs(_components.p,{children:[`The canonical input is `,_jsxs(_components.strong,{children:[`mono 16-bit PCM (`,_jsx(_components.code,{children:`s16le`}),`)`]}),` at the sample rate configured for the avatar view. Convert source audio first when its channels, encoding, or sample rate do not match.`]}),`\n`,_jsx(_components.p,{children:`The audio format belongs to each avatar view, not to SDK initialization, so two views can run different sample rates. It can also be changed on a live view; a change interrupts the current turn and, in Direct Mode, disconnects, so reconnect before sending audio in the new format.`}),`\n`,_jsx(_components.p,{children:`Some SDK entry points also accept Opus. Format support and configuration differ by platform and integration, so use the exact SDK reference rather than mixing PCM and Opus within one session:`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.a,{href:`/sdk-reference/web-sdk/reference#audioformat`,children:`Web`}),` · `,_jsx(_components.a,{href:`/sdk-reference/ios-sdk/api-reference#audioformat`,children:`iOS`}),` · `,_jsx(_components.a,{href:`/sdk-reference/android-sdk/api-reference#audioformat`,children:`Android`}),` · `,_jsx(_components.a,{href:`/sdk-reference/flutter-sdk/api-reference#audioformat`,children:`Flutter`})]}),`\n`,_jsx(Heading,{level:`2`,id:`end-input-or-interrupt`,children:`End input or interrupt`}),`\n`,_jsx(_components.p,{children:`These actions have different meanings:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`End input`}),` marks the final chunk of the current speech turn. Motion Server can finalize that turn, and AvatarKit continues playing the buffered result.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Interrupt`}),` cancels the active turn and clears queued audio and motion data. Use it for barge-in or when the response should stop immediately.`]}),`\n`]}),`\n`,_jsx(_components.p,{children:`Do not use interruption as the normal end marker for every response.`})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{AudioSendTimingDiagram,default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Audio","sidebarTitle":"Audio","description":"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.","href":"/concepts/audio"}}},"mdxSourceWithNoJs":null,"noJsSplices":[[161,11200,"null;function _creat"]]},"mdxExtracts":{"tableOfContents":[{"title":"Send timing","slug":"send-timing","depth":2,"children":[{"title":"If your source is paced","slug":"if-your-source-is-paced","depth":3,"children":[]}]},{"title":"Input format","slug":"input-format","depth":2,"children":[]},{"title":"End input or interrupt","slug":"end-input-or-interrupt","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/_mintlify/custom/spatius/842af4b2cdb9f6f1/ga4.js"],"subdomain":"spatius","siteName":"Spatius","title":"Audio - Spatius","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-10-02T19:38:08.995Z","originalFileLocation":"concepts/audio.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Spatius\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Audio - Spatius\">\u003cmeta data-mint-head property=\"og:description\" content=\"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Spatius\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.spatius.ai/concepts/audio\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://spatius.mintlify.site/_mintlify/api/og/v2?division=Concepts\u0026amp;title=Audio\u0026amp;description=How+avatar+speech+audio+is+formatted%2C+sent%2C+finalized%2C+and+interrupted+for+synchronized+Avatar+playback.\u0026amp;theme=3c6d6186e4656af8ad36b894\u0026amp;signature=38736dcd481750761e1c6deafe2eec832b4dda4da05a87c3f3b7515636c94afd\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Audio - Spatius\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://spatius.mintlify.site/_mintlify/api/og/v2?division=Concepts\u0026amp;title=Audio\u0026amp;description=How+avatar+speech+audio+is+formatted%2C+sent%2C+finalized%2C+and+interrupted+for+synchronized+Avatar+playback.\u0026amp;theme=3c6d6186e4656af8ad36b894\u0026amp;signature=38736dcd481750761e1c6deafe2eec832b4dda4da05a87c3f3b7515636c94afd\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/spatius/w2LH48fbTgqvj-Bh/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Spatius\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#9399E3\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.spatius.ai/concepts/audio\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/concepts/audio.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.spatius.ai/#organization\",\"name\":\"Spatius\",\"url\":\"https://docs.spatius.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/spatius/wOjIDpP8OnEpRngp/images/spatius-logo-wordmark-black.png?fit=max\u0026auto=format\u0026n=wOjIDpP8OnEpRngp\u0026q=85\u0026s=3a9b7965a4bcfae37a65eefa457011ba\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.spatius.ai#website\",\"name\":\"Spatius\",\"url\":\"https://docs.spatius.ai\",\"publisher\":{\"@id\":\"https://docs.spatius.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.spatius.ai/concepts/audio#webpage\",\"url\":\"https://docs.spatius.ai/concepts/audio\",\"name\":\"Audio\",\"description\":\"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\",\"dateModified\":\"2026-10-02T19:38:08.995Z\",\"isPartOf\":{\"@id\":\"https://docs.spatius.ai#website\"},\"breadcrumb\":{\"@id\":\"https://docs.spatius.ai/concepts/audio#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.spatius.ai/concepts/audio#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Concepts\",\"item\":\"https://docs.spatius.ai/concepts/avatar\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Audio\",\"item\":\"https://docs.spatius.ai/concepts/audio\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://docs.spatius.ai/concepts/audio#article\",\"headline\":\"Audio\",\"name\":\"Audio\",\"description\":\"How avatar speech audio is formatted, sent, finalized, and interrupted for synchronized Avatar playback.\",\"url\":\"https://docs.spatius.ai/concepts/audio\",\"mainEntityOfPage\":{\"@id\":\"https://docs.spatius.ai/concepts/audio#webpage\"},\"dateModified\":\"2026-10-02T19:38:08.995Z\",\"publisher\":{\"@id\":\"https://docs.spatius.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.spatius.ai#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Spatius\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"Spatius\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/concepts/audio.md","siteName":"Spatius","nav":{"global":null,"tabs":[{"tab":"Guides","groups":[{"group":"Start Here","icon":"rocket","pages":[{"title":"Welcome to Spatius","sidebarTitle":"Welcome","href":"/getting-started"},{"title":"How It Works","sidebarTitle":"How It Works","href":"/concepts/how-it-works"}]},{"group":"Quickstarts","icon":"bolt","pages":[{"title":"Web Quickstart","sidebarTitle":"Web","href":"/quickstarts/web-sdk"},{"title":"iOS Quickstart","sidebarTitle":"iOS","href":"/quickstarts/ios-sdk"},{"title":"Android Quickstart","sidebarTitle":"Android","href":"/quickstarts/android-sdk"}]},{"group":"Integrations","icon":"plug","pages":[{"title":"Integrations","sidebarTitle":"Overview","href":"/integrations/overview"},{"group":"LiveKit Agents","tag":"Recommended","pages":[{"title":"LiveKit Agents Integration","sidebarTitle":"Overview","href":"/livekit-agents/overview"},{"title":"Set Up LiveKit Agents Integration","sidebarTitle":"Agent setup","href":"/livekit-agents/server"},{"title":"LiveKit Agents Integration Client","sidebarTitle":"Client","href":"/livekit-agents/client"}]},{"group":"Agora ConvoAI","tag":"Recommended","pages":[{"title":"Agora Convo AI Integration","sidebarTitle":"Overview","href":"/agora-convoai/overview"},{"group":"Agent setup","pages":[{"title":"Agora Convo AI Agent","sidebarTitle":"Convo AI Agent","href":"/agora-convoai/convo-ai-agent"},{"title":"TEN Framework Extension","sidebarTitle":"TEN Extension","href":"/agora-convoai/ten-extension"}]},{"title":"Agora Convo AI Client","sidebarTitle":"Client","href":"/agora-convoai/client"}]},{"group":"Direct Mode","pages":[{"title":"Direct Mode Client","sidebarTitle":"Client","href":"/direct-mode/client"}]}]},{"group":"Concepts","icon":"lightbulb","pages":[{"title":"Avatar","sidebarTitle":"Avatar","href":"/concepts/avatar"},{"title":"Avatar Background","sidebarTitle":"Background","description":"Download the optional 16:9 Avatar background from Spatius Studio and render it behind AvatarKit in your app.","href":"/concepts/avatar-background"},{"title":"Audio","sidebarTitle":"Audio","href":"/concepts/audio"},{"title":"Client Lifecycle","sidebarTitle":"Client Lifecycle","description":"The AvatarKit client lifecycle from initialization and asset loading through rendering, interaction, and cleanup.","href":"/concepts/lifecycle"},{"title":"Client State \u0026 Events","sidebarTitle":"Client State \u0026 Events","href":"/concepts/state-events"}]},{"group":"Pricing","icon":"credit-card","pages":[{"title":"Custom Avatar Creation Pricing","sidebarTitle":"Custom Avatar Creation","href":"/pricing/custom-avatar-creation"},{"title":"Realtime Avatar Pricing","sidebarTitle":"Realtime Avatar","href":"/pricing/realtime-avatar"},{"title":"Video Generation Pricing","sidebarTitle":"Video Generation","href":"/pricing/video-generation"}]},{"group":"Examples \u0026 Support","icon":"book","pages":[{"title":"Demo Projects","sidebarTitle":"Demo Projects","href":"/resources/demo-projects"},{"title":"FAQ","href":"/resources/faq"}]}]},{"tab":"SDK Reference","pages":[{"title":"SDK Reference","href":"/reference/overview"}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/api-reference/overview"}]}]},"basePath":"","title":"Audio - Spatius","deploymentId":"dpl_vSEmv961Ncyvz1psVwgUnTXQMSYk","buildId":"dpl_vSEmv961Ncyvz1psVwgUnTXQMSYk","dynamics":["07c022d89b"],"chunks":["Frame-BRsfsIIG.js","DynamicMarkdown-NFqqduU7.js"]}