通过逐轮策略优化，让大模型更懂用户情绪需求。
Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter
- 用蒙特卡洛树搜索构建分步策略数据集，精准匹配对话每轮情绪支持策略。
- 在三款主流模型上测试，策略选择准确率显著提升，偏见问题明显缓解。
- 适合需要高情商对话能力的客服、心理咨询等场景应用。
现代社会情绪压力上升，对情感支持对话（ESC）的需求增加。尽管大语言模型（LLMs）在该领域展现出潜力，但仍面临两大挑战：策略选择准确率低和偏好偏差，限制了其对用户情绪需求的适应性。现有监督微调（SFT）方法难以解决这些问题，因其仅基于单一标准答案进行刚性训练，未建模策略间的细微权衡。为此，我们提出链式策略优化（CSO），在每轮对话中优化策略选择偏好。首先利用蒙特卡洛树搜索构建高质量的偏好数据集 ESC-Pro，包含逐轮策略-回复对。在 ESC-Pro 上使用 CSO 进行训练，显著提升了策略准确率并缓解了偏差，使模型生成更具同理心和上下文相关性的回应。在 LLaMA-3.1-8B、Gemma-2-9B 与 Qwen2.5-7B 上的实验表明，CSO 显著优于标准 SFT，验证了细粒度、逐轮偏好建模在情感支持对话中的有效性。
原文摘要 · Abstract (English)
The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to emotional needs of users. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose Chain-of-Strategy Optimization (CSO), a novel approach that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turn-level strategy-response pairs. Training on ESC-Pro with CSO improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。