提升语音助手对第三方打断的鲁棒性，让其能准确分辨说话人。
Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions
- 构建8.8万条带说话人感知难例的数据集，强化声学线索优先
- 实验表明新方法有效避免模型依赖语义而忽略声音特征
- 适合研究语音交互、多说话人场景鲁棒性的学者使用
尽管近期语音语言模型（SLMs）已在真实场景中广泛应用，但它们缺乏区分第三方打断与主用户连续对话的能力，易导致上下文失效。为此，我们提出TPI-Train数据集，包含88,000个实例，采用说话人感知的难例设计，强制模型优先关注声学线索以处理打断；同时构建TPI-Bench评估框架，用于严格衡量模型在欺骗性情境下的打断处理策略和精确说话人识别能力。实验表明，该数据集设计有效缓解了语义捷径学习——即模型过度依赖语义上下文而忽视关键声学信号的问题。我们认为本工作为克服SLMs中以文本为主导的单模态依赖提供了基础资源，推动更鲁棒的多方语音交互发展。代码已公开于https://tpi-va.github.io。
原文摘要 · Abstract (English)
While recent Spoken Language Models (SLMs) have been actively deployed in real-world scenarios, they lack the capability to discern Third-Party Interruptions (TPI) from the primary user's ongoing flow, leaving them vulnerable to contextual failures. To bridge this gap, we introduce TPI-Train, a dataset of 88K instances designed with speaker-aware hard negatives to enforce acoustic cue prioritization for interruption handling, and TPI-Bench, a comprehensive evaluation framework designed to rigorously measure the interruption-handling strategy and precise speaker discrimination in deceptive contexts. Experiments demonstrate that our dataset design mitigates semantic shortcut learning-a critical pitfall where models exploit semantic context while neglecting acoustic signals essential for discerning speaker changes. We believe our work establishes a foundational resource for overcoming text-dominated unimodal reliance in SLMs, paving the way for more robust multi-party spoken interaction. The code for the framework is publicly available at https://tpi-va.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。