arXiv:2510.05150cs.CLcs.AI2025-10中稿 · SIGDIAL 2026被引 16

让对话模型像人一样边听边思考,提升实时互动质量。

Chronological Thinking in Full-Duplex Spoken Dialogue Language Models

  • 模型在听用户说话时持续进行因果推理,不提前预判。
  • 推理过程不增加延迟,用户停说后立即回应。
  • 适合需要实时交互的语音助手、客服系统等场景。

近期语音对话语言模型(SDLMs)的发展推动了从轮次式向全双工系统的转变,即模型在持续接收用户语音流的同时生成回应。这种同时听与说的设计支持实时交互,并能处理用户抢话等动态行为。然而,现有系统在听觉阶段仅反复预测静音标记,使模型处于空闲状态,这与人类在对话中轻度思考的行为不符。为此,我们提出「时间顺序思考」机制,旨在提升全双工对话模型的响应质量。该机制具有三大特性:(1)严格因果性:模型在聆听过程中逐步推理,仅基于过去音频更新内部假设,无前瞻;(2)无额外延迟:推理在聆听窗口内完成,用户停止讲话后立即开始回应,无需等待。实验表明,该方法在客观指标和人工评估中均显著提升响应质量,且对复杂对话动态具有鲁棒性,在全双工交互指标上表现优异。

原文摘要 · Abstract (English)

Recent advances in spoken dialogue language models (SDLMs) reflect growing interest in shifting from turn-based to full-duplex systems, where the models continuously perceive user speech streams while generating responses. This simultaneous listening and speaking design enables real-time interaction and the agent can handle dynamic conversational behaviors like user barge-in. However, during the listening phase, existing systems keep the agent idle by repeatedly predicting the silence token, which departs from human behavior: we usually engage in lightweight thinking during conversation rather than remaining absent-minded. Inspired by this, we propose Chronological Thinking, an on-the-fly conversational thinking mechanism that aims to improve response quality in full-duplex SDLMs. Specifically, chronological thinking presents a paradigm shift from conventional LLM thinking approaches, such as Chain-of-Thought, purpose-built for streaming acoustic input. (1) Strictly causal: the agent reasons incrementally while listening, updating internal hypotheses only from past audio with no lookahead. (2) No additional latency: reasoning is amortized during the listening window; once the user stops speaking, the agent halts thinking and begins speaking without further delay. Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations show consistent improvements in response quality. Furthermore, chronological thinking robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.

语音对话全双工思考机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。