用状态机机制让语音助手实时听懂用户打断，低延迟高效处理。
TurnFSM for Full-Duplex Dialogue System: Internalizing State-Machine Logic for Streaming Semantic Voice Activity Detection and Utterance-Level Rejection
- 将对话状态转为显式有限状态机，统一处理语音活动检测与语句拒绝
- 在低延迟流式场景下，性能接近专用模型，且无额外推理开销
- 首次采用一阶状态转移，避免历史状态累积，适合资源受限设备
全双工语音助手需在发声时持续监听，应对用户打断，面临低延迟与资源受限的挑战。现有端到端模型在语音域适配后会削弱推理能力，而级联流水线引入额外推理开销和手工控制逻辑。我们提出 TurnFSM，一种基于大语言模型的状态预测框架，将对话轮次控制内化为显式的有限状态转换，统一实现流式语义语音活动检测（VAD）与语句级拒绝。TurnFSM 将提交与拒绝分解为串行决策流程，降低多任务干扰，同时保持与单任务模型相当的性能。我们进一步引入一阶状态转移机制，在训练中仅依赖前一状态，使推理可使用标准因果掩码与原始 LLM 位置编码，避免历史状态令牌累积和冗余逐步状态生成。实验表明，TurnFSM 持续优于二分类头基线，并在性能上与任务特定模型相当。
原文摘要 · Abstract (English)
Full-duplex voice assistants must continuously listen while speaking, handling user interruptions under low-latency and resource-constrained streaming conditions. Existing end-to-end full-duplex models can compromise reasoning-related capabilities after speech-domain adaptation, whereas cascaded pipelines introduce extra inference overhead and handcrafted control logic. We propose TurnFSM, an LLM-based state prediction framework that internalizes turn control as explicit finite-state transitions, unifying streaming semantic VAD and utterance-level rejection. TurnFSM decomposes submission and rejection into a serial decision process, reducing multi-task interference while maintaining performance comparable to single-task models. We further introduce a first-order state transition mechanism that enforces the dependency on only the previous state during training, enabling compact inference with the standard causal mask and original LLM positional encoding while avoiding historical state-token accumulation and unnecessary step-by-step state generation. Experimental results show that TurnFSM consistently outperforms the binary-head baseline and remains competitive with task-specific models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。