检测对话中用户主动求助修复的信号，提升聊天机器人理解力。
"Mm, Wat?" Detecting Other-initiated Repair Requests in Dialogue
- 融合语言与语调特征，识别用户求助信号
- 语调线索使模型准确率显著提升
- 适合对话系统优化与人机交互研究者
保持相互理解是人类对话避免中断的关键，其中修复机制（特别是当一方发出问题信号并提示另一方解决时的其他发起修复，OIR）起着核心作用。然而，当前对话代理仍难以识别用户发起的修复请求，导致对话中断或用户流失。本文提出一种多模态模型，通过整合基于会话分析的语言与语调特征，自动检测荷兰语对话中的修复请求。结果表明，语调线索可有效补充语言特征，显著提升预训练文本与音频嵌入模型的表现，揭示了不同特征间的协同机制。未来方向包括引入视觉线索，探索多语言及跨场景语料库以评估模型的鲁棒性与泛化能力。
原文摘要 · Abstract (English)
Maintaining mutual understanding is a key component in human-human conversation to avoid conversation breakdowns, in which repair, particularly Other-Initiated Repair (OIR, when one speaker signals trouble and prompts the other to resolve), plays a vital role. However, Conversational Agents (CAs) still fail to recognize user repair initiation, leading to breakdowns or disengagement. This work proposes a multimodal model to automatically detect repair initiation in Dutch dialogues by integrating linguistic and prosodic features grounded in Conversation Analysis. The results show that prosodic cues complement linguistic features and significantly improve the results of pretrained text and audio embeddings, offering insights into how different features interact. Future directions include incorporating visual cues, exploring multilingual and cross-context corpora to assess the robustness and generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。