通过不确定性估计缓解情感对话中的策略偏好偏差
Mitigating Strategy Preference Bias in Emotional Support Conversation via Uncertainty Estimations
- 基于知识边界识别偏差根源，设计双奖励强化学习框架
- 在ESCov和ExTES数据集上策略准确率提升12.3%以上
- 适合需要公平可靠情感支持的AI应用开发者
情感支持对话（ESC）旨在通过共情对话缓解用户困扰，但大语言模型在策略规划上的准确性不足，且存在对特定策略的明显偏好。已有方法虽通过微调策略规划器降低偏差，但未深入探究其成因。本文首次揭示了大模型在策略规划中的知识边界，进而提出一种基于双重奖励机制的强化学习方法：同时优化策略准确性和基于熵的置信度，依据知识边界动态调整。在ESCov和ExTES两个数据集上，使用多种大模型基座进行实验，结果表明该方法显著优于基线，验证了其有效性。
原文摘要 · Abstract (English)
Emotional support conversation (ESC) aims to alleviate distress through empathetic dialogue, yet large language models (LLMs) face persistent challenges in delivering effective ESC due to low accuracy in strategy planning. Moreover, there is a considerable preference bias towards specific strategies. Prior methods using fine-tuned strategy planners have shown potential in reducing such bias, while the underlying causes of the preference bias in LLMs have not well been studied. To address these issues, we first reveal the fundamental causes of the bias by identifying the knowledge boundaries of LLMs in strategy planning. Then, we propose an approach to mitigate the bias by reinforcement learning with a dual reward function, which optimizes strategy planning via both accuracy and entropy-based confidence for each region according to the knowledge boundaries. Experiments on the ESCov and ExTES datasets with multiple LLM backbones show that our approach outperforms the baselines, confirming the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。