用推理链+强化学习提升长文本心理支持的共情能力
Empathy-R1: A Chain-of-Empathy and Reinforcement Learning Framework for Long-Form Mental Health Support
- 构建共情推理链,分步分析求助者情绪、成因与意图
- 在中文心理支持数据集上实现44.30%胜率领先基线
- 响应更可解释且符合语境,适合真实心理咨询场景
共情对有效心理支持至关重要,尤其在处理长篇咨询文本(LCTs)时。现有大语言模型虽语义流畅,但在中国语境下缺乏结构化推理能力。为此,我们提出Empathy-R1框架,融合共情推理链(CoE)与强化学习(RL),提升长文本回复质量。受认知行为疗法启发,CoE引导模型分步推理求助者的情绪、原因和意图,使思维过程透明可解释。框架基于新构建的大规模中文数据集Empathy-QA,采用两阶段训练:先通过监督微调建立推理结构,再用奖励模型指导强化学习优化回应的治疗相关性与语境适切性。实验表明,Empathy-R1在关键自动指标上表现优异;更重要的是,人工评估确认其优势,在新基准上达到44.30%的胜率。该方法使AI能生成更具可解释性与语境敏感性的回应,显著推进负责任的心理健康支持AI发展。
原文摘要 · Abstract (English)
Empathy is critical for effective mental health support, especially when addressing Long Counseling Texts (LCTs). However, existing Large Language Models (LLMs) often generate replies that are semantically fluent but lack the structured reasoning necessary for genuine psychological support, particularly in a Chinese context. To bridge this gap, we introduce Empathy-R1, a novel framework that integrates a Chain-of-Empathy (CoE) reasoning process with Reinforcement Learning (RL) to enhance response quality for LCTs. Inspired by cognitive-behavioral therapy, our CoE paradigm guides the model to sequentially reason about a help-seeker's emotions, causes, and intentions, making its thinking process both transparent and interpretable. Our framework is empowered by a new large-scale Chinese dataset, Empathy-QA, and a two-stage training process. First, Supervised Fine-Tuning instills the CoE's reasoning structure. Subsequently, RL, guided by a dedicated reward model, refines the therapeutic relevance and contextual appropriateness of the final responses. Experiments show that Empathy-R1 achieves strong performance on key automatic metrics. More importantly, human evaluations confirm its superiority, showing a clear preference over strong baselines and achieving a Win@1 rate of 44.30% on our new benchmark. By enabling interpretable and contextually nuanced responses, Empathy-R1 represents a significant advancement in developing responsible and genuinely beneficial AI for mental health support.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。