让语音助手实时说话、规划、调用工具，不卡顿也不等用户说完。
DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action
- 用160毫秒共享时间轴同步处理语音、语言和动作指令。
- 支持边说边计划、边说边调用工具，还能自然打断和回应。
- 适合做智能客服、语音助手等需要实时交互的场景。
近年来，语音对话模型从轮次制转向全双工设计，即模型持续聆听用户的同时生成回复。然而，现有全双工架构仍缺乏内置的对话规划与工具调用通道，导致实时智能行为要么受限于轮次边界，要么依赖外部串联流程。我们提出 DuplexSLA，一种原生全双工语音-语言-动作基础模型，能够在统一的160毫秒时隙上解码助理语音与结构化动作流。该模型基于双流三通道结构：连续用户音频通道、离散助理音频通道，以及速率受限的文本动作通道，均由单一主干网络联合解码，使聆听、说话、规划与工具调用在同一个时间钟上协同进行。其核心能力包括：(1) 语义驱动的轮次控制，中断、停顿与反馈信号由模型内部处理，无需外部语义语音活动检测（VAD）；(2) 对话中规划与工具调用，规划文本与结构化工具调用在动作通道输出，不中断语音生成，实现多动作与反馈触发的工具使用与语音交错。为综合评估这些能力，我们构建了 DuplexSLA-Bench，一个涵盖停顿、打断、反馈轮次控制及三种风格对话中工具调用的全双工基准。项目主页、互动演示与评估套件已开源：https://github.com/hyzhang24/DuplexSLA。
原文摘要 · Abstract (English)
Recent advances in spoken dialogue language models have shifted from turn-based to full-duplex designs, where the model continuously listens to the user while generating responses. However, existing duplex backbones still lack a native channel for in-conversation planning and tool calling, leaving real-time agentic behaviour either tied to turn boundaries or relegated to an external cascade. We propose DuplexSLA, a native full-duplex Speech-Language-Action foundation model that decodes assistant audio together with a structured action stream on a shared 160 ms chunk timeline. DuplexSLA is built on a dual-stream three-channel formulation: a continuous user audio channel, a discrete assistant audio channel, and a rate-limited textual action channel, all decoded jointly by a single backbone, so that listening, speaking, planning, and tool calling unfold on one shared clock. Two capabilities define the model: (1) semantic-driven turn-taking control, where interruption, pause, and backchannel are handled inside the same backbone instead of by an external semantic VAD; and (2) in-conversation planning and tool calling, where planning text and structured tool calls are emitted on the action channel without halting assistant audio, so that multi-action and backchannel-triggered tool use are interleaved with ongoing speech. To evaluate these capabilities together, we further construct DuplexSLA-Bench, a duplex benchmark covering pause, interrupt, and backchannel turn-taking together with three styles of in-conversation tool calling. Our project page, interactive demos, and the DuplexSLA-Bench evaluation suite are publicly available at https://github.com/hyzhang24/DuplexSLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。