arXiv:2604.17188cs.CLcs.AI2026-04ACL

用推理和人类偏好优化对话摘要,更准确忠实。

Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization

论文配图:Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization
图 1 · 摘自论文原文
  • 先学大模型的推理过程,再用奖励机制优化摘要。
  • 在多角色对话上,事实准确率显著提升。
  • 适合需要高可信度摘要的研究者或应用者。

多角色对话摘要需捕捉多说话人之间的复杂互动,同时保留角色特定信息与事实一致性。然而,现有方法主要优化ROUGE和BERTScore等自动指标,倾向于表面模仿参考文本,而非真正提升忠实度或符合人类偏好。本文提出一种新框架,将显式的认知式推理与基于奖励的优化结合。首先从大模型中提炼结构化推理轨迹(如分步推断与中间反思),作为辅助监督信号,通过分阶段有监督微调初始化一个具备推理意识的摘要模型。随后采用GRPO算法,设计双重原则奖励函数,融合指标信号与人类对关键信息覆盖、隐含推断、事实忠实性和简洁性的标准。在多语言多角色对话基准测试中,该方法在ROUGE和BERTScore上达到强基线水平。具体地,CSDS数据集验证了语义一致性的稳定性,而SAMSum分析显示事实忠实度和模型偏好对齐均有明显提升。结果表明,推理感知与偏好感知训练对可靠对话摘要至关重要。代码与数据集见https://huggingface.co/collections/NebulaPixel/summorchestra-multirole-summary。

原文摘要 · Abstract (English)

Multi-role dialogue summarization requires modeling complex interactions among multiple speakers while preserving role-specific information and factual consistency. However, most existing methods optimize for automatic metrics such as ROUGE and BERTScore, which favor surface-level imitation of references rather than genuine gains in faithfulness or alignment with human preferences. We propose a novel framework that couples explicit cognitive-style reasoning with reward-based optimization for multi-role dialogue summarization. Our method first distills structured reasoning traces (e.g., step-by-step inferences and intermediate reflections) from a large teacher model and uses them as auxiliary supervision to initialize a reasoning-aware summarizer via staged supervised fine-tuning. It then applies GRPO with a dual-principle reward that blends metric-based signals with human-aligned criteria targeting key information coverage, implicit inference, factual faithfulness, and conciseness. Experiments on multilingual multi-role dialogue benchmarks show that our method matches strong baselines on ROUGE and BERTScore. Specifically, results on CSDS confirm the framework's stability in semantic consistency, while in-depth analysis on SAMSum demonstrates clear gains in factual faithfulness and model-based preference alignment. These findings underscore the value of reasoning-aware and preference-aware training for reliable dialogue summarization. Checkpoints and datasets are available at https://huggingface.co/collections/NebulaPixel/summorchestra-multirole-summary.

对话摘要推理增强偏好学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。