让语音助手学会在多人对话中判断何时该说话，避免干扰
Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue
- 基于对话上下文决定是否发言，而非仅依赖沉默信号
- 8个大模型零样本表现差，准确率提升最高达23个百分点
- 适合需要自然交互的多角色语音助手场景
现有语音助手将每个检测到的停顿都视为发言机会，这在双人对话中有效，但在多人对话中，停顿频繁且含义模糊。若助手在每次停顿都发言，反而会破坏对话流程。本文提出上下文感知的轮换策略：在每次检测到停顿后，结合完整对话上下文判断助手是否应发言。我们构建了一个包含超过12万条标注对话的基准数据集，涵盖三个多人对话语料库。评估八个近期大型语言模型发现，它们在零样本提示下均无法完成上下文感知的轮换。我们提出一种带推理轨迹的监督微调方法，使平衡准确率最高提升23个百分点。结果表明，上下文感知轮换并非涌现能力，必须显式训练。
原文摘要 · Abstract (English)
Existing voice AI assistants treat every detected pause as an invitation to speak. This works in dyadic dialogue, but in multi-party settings, where an AI assistant participates alongside multiple speakers, pauses are abundant and ambiguous. An assistant that speaks on every pause becomes disruptive rather than useful. In this work, we formulate context-aware turn-taking: at every detected pause, given the full conversation context, our method decides whether the assistant should speak or stay silent. We introduce a benchmark of over 120K labeled conversations spanning three multi-party corpora. Evaluating eight recent large language models, we find that they consistently fail at context-aware turn-taking under zero-shot prompting. We then propose a supervised fine-tuning approach with reasoning traces, improving balanced accuracy by up to 23 percentage points. Our findings suggest that context-aware turn-taking is not an emergent capability; it must be explicitly trained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。