arXiv:2606.23124cs.CLcs.AI2026-06

用专家心理标注训练小模型,让其学会共情对话。

PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation

论文配图:PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
图 1 · 摘自论文原文
  • 训练时引入专家标注等特权信息,指导小模型学习共情逻辑。
  • 在多个数据集上表现优于同类压缩方法,部分指标超越大模型。
  • 适合需要低资源部署共情对话系统的团队使用。

大型语言模型在生成多样且情境敏感的共情对话方面展现出显著能力,但其计算开销严重限制了在资源受限环境中的部署。虽然知识蒸馏提供了有前景的压缩方案,但通常无法传递共情所必需的细微理解,因为它忽略了引导人际连接的隐含上下文线索。为此,我们提出一种特权信息增强的知识蒸馏方法(PRIDE),用于共情对话生成。该方法利用仅在训练阶段可用的特权信息(如专家心理标注或未来事件摘要),在推理阶段无需额外输入即可将教师模型的共情推理能力转移给小型模型。PRIDE包含三个关键组件:(1) 共情推理提示,引导教师模型逐步分解情感理解与情境分析过程;(2) 多源注意力机制,使学生模型能有效融合特权信息;(3) 双重对齐损失,结合反向KL散度与最大均值差异,确保在logit和特征层面实现稳健的知识迁移。在多模态和纯文本数据集上的实验表明,该方法性能优异,在某些情况下甚至达到或超过更大教师模型在准确性和语义相关性方面的表现。

原文摘要 · Abstract (English)

Large language models have demonstrated significant capabilities in generating diverse and context-aware responses for empathetic dialogue. However, their computational demands severely limit their deployment in resource-constrained environments. While knowledge distillation offers a promising compression solution, it often fails to transfer the nuanced understanding essential for empathy, as it overlooks the implicit contextual cues that guide human connection. To bridge this gap, we propose a \textbf{pr}ivileged \textbf{i}nformation-enhanced knowledge \textbf{d}istillation method for \textbf{e}mpathetic dialogue generation (PRIDE). Our method leverages privileged information, such as expert psychological annotations or future event summaries, which is available exclusively during training but unavailable at inference time. This allows us to transfer the teacher model's empathetic reasoning to smaller models without relying on extra inputs during deployment. Specifically, PRIDE has three key components: (1) An empathy-reasoning prompt that guides the teacher to explicitly decompose the empathetic process into understanding feelings and analyzing situations step-by-step; (2) A multi-source attention mechanism that directs the student to effectively integrate privileged information; (3) A dual-alignment loss that combines reversed Kullback-Leibler divergence and maximum mean discrepancy to ensure robust knowledge transfer at both logit and feature levels. Experiments on multi-modal and text-only datasets demonstrate that our method achieves competitive performance, and in some cases matches or even surpasses larger teacher models in terms of accuracy and semantic relevance.

共情对话知识蒸馏小模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。