用思维链让语音对话系统学会共情，无需问答数据。
Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data
- 两阶段训练：先听懂语音内容与情绪，再用思维链生成共情回复。
- 在无问答数据下仍能生成高共情度回复，提升用户满意度。
- 适合做情感化语音助手、心理咨询机器人等场景。
共情对话对自然的人机交互至关重要，能让对话系统以更个性化和情感化的方式回应，从而提升用户满意度和参与度。大语言模型（LLMs）的出现彻底改变了对话生成，展现出在多模态领域中的潜力。许多研究将语音与文本型LLM结合，以语音问题为输入，输出文本回复。然而，缺乏包含语音风格信息的语音问答数据用于监督微调（SFT），限制了这些系统的性能。结果是，尽管这些系统能理解语音内容，却常难以生成共情回复。为此，我们提出一种新方法——听、感知、表达（LPE），无需问答数据。该方法采用两阶段训练：首先引导LLM聆听语音内容并感知情绪，随后利用思维链（CoT）提示激发模型基于所听内容与情绪线索生成共情回复。实验验证了该方法的有效性。据我们所知，这是首个将思维链应用于语音对话的尝试。
原文摘要 · Abstract (English)
Empathetic dialogue is crucial for natural human-computer interaction, allowing the dialogue system to respond in a more personalized and emotionally aware manner, improving user satisfaction and engagement. The emergence of large language models (LLMs) has revolutionized dialogue generation by harnessing their powerful capabilities and shown its potential in multimodal domains. Many studies have integrated speech with text-based LLMs to take speech question as input and output text response. However, the lack of spoken question-answering datasets that include speech style information to supervised fine-tuning (SFT) limits the performance of these systems. As a result, while these systems excel at understanding speech content, they often struggle to generate empathetic responses. In response, we propose a novel approach that circumvents the need for question-answering data, called Listen, Perceive, and Express (LPE). Our method employs a two-stage training process, initially guiding the LLM to listen the content and perceive the emotional aspects of speech. Subsequently, we utilize Chain-of-Thought (CoT) prompting to unlock the model's potential for expressing empathetic responses based on listened spoken content and perceived emotional cues. We employ experiments to prove the effectiveness of proposed method. To our knowledge, this is the first attempt to leverage CoT for speech-based dialogue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。