用强化学习让AI更懂情绪和上下文,生成更专业的心理支持对话。
Context-Emotion Aware Therapeutic Dialogue Generation: A Multi-component Reinforcement Learning Approach to Language Models for Mental Health Support
- 重构输入格式,同时处理用户语境和情绪状态。
- 奖励函数结合专业治疗逻辑,提升情感准确率至99.34%。
- 适合心理治疗辅助系统开发,兼顾隐私与临床有效性。
心理健康障碍带来巨大全球社会经济负担。尽管大语言模型(LLMs)可提供全天候、无评判的互动以弥补这一缺口,但预训练模型在生成治疗对话时缺乏情境连贯性和情感一致性。现有方法存在三大缺陷:1)监督微调(SFT)导致输出重复且不敏感于上下文,难以平衡临床准确性与真实共情;2)基于强化学习(RL)的系统依赖通用奖励函数(如BLEU、ROUGE),优先考虑词汇相似度而非临床特定的情感适宜性与上下文相关性;3)LLMs资源消耗大且存在数据隐私风险,使本地部署在临床场景中不可行。本研究探索将SFT与RL技术应用于GPT-2,以增强其治疗对话生成能力。方法上重构输入格式,实现上下文与情绪状态的同步处理,并设计新型多组件奖励函数,明确对齐模型输出与专业治疗逻辑及标注情绪。结果表明,相比基线GPT-2,RL在多项评估指标上显著提升:BLEU(0.0111)、ROUGE-1(0.1397)、ROUGE-2(0.0213)、ROUGE-L(0.1317)、METEOR(0.0581)。人工评估确认高上下文相关性与专业性,且RL达到99.34%的情绪准确率,远高于基线的66.96%。结果证明,强化学习能有效构建可作为治疗师辅助工具的对话系统,同时保持必要的人类临床监督。
原文摘要 · Abstract (English)
Mental health disorders impose a substantial global socioeconomic burden. While large language models (LLMs) offer 24/7, non-judgmental interactions to address this gap, pretrained models lack contextual coherence and emotional alignment for appropriate therapeutic dialogue. Existing methods suffer from three critical methodological gaps: 1) Supervised Fine-Tuning (SFT) produces repetitive, context-insensitive outputs that fail to balance clinical accuracy with genuine empathy; 2) Reinforcement Learning (RL)-based therapeutic systems rely on generic reward functions (e.g., BLEU, ROUGE) that prioritise lexical similarity over clinical-specific emotional appropriateness and contextual relevance; 3) LLMs are resource-intensive and pose data privacy risks, making local deployment in clinical settings infeasible. To address these gaps, this study investigates the application of SFT and RL techniques to enhance GPT-2's capacity for therapeutic dialogue generation. The methodology restructured input formats to enable simultaneous processing of contextual information and emotional states alongside user input, employing a novel multi-component reward function that explicitly aligns model outputs with professional therapeutic logic (not just lexical overlap) and annotated emotions. Results demonstrated substantial improvements through RLs over baseline GPT-2 across multiple evaluation metrics: BLEU (0.0111), ROUGE-1 (0.1397), ROUGE-2 (0.0213), ROUGE-L (0.1317), and METEOR (0.0581). LLM evaluation confirmed high contextual relevance and professionalism, while RL achieved 99.34% emotion accuracy compared to 66.96% for baseline GPT-2. These findings demonstrate RL's effectiveness in developing therapeutic dialogue systems that can serve as valuable assistive tools for therapists, while maintaining essential human clinical oversight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。