用对比学习减少摘要幻觉，让模型生成更一致的内容。
BRIDO: Bringing Democratic Order to Abstractive Summarization
- 通过对比学习鼓励相似候选摘要，抑制幻觉内容
- 在XSum和CNN/DM数据集上一致性得分提升6.25%和3.82%
- 适合关注摘要可靠性与一致性的研究者使用
幻觉指大语言模型生成的不准确、无关或不一致文本。尽管大模型在诸多任务中表现优异，但幻觉仍是实际应用中的主要挑战。本文针对抽象摘要中的幻觉问题，通过缓解暴露偏差来改进。现有方法如BRIO旨在提升ROUGE分数，而我们提出的新方法采用类似暴露偏差缓解策略，但目标转向减少幻觉。我们推测：在一组候选输出中，含幻觉的内容占少数，且与其他输出相似度较低。因此，本方法利用对比学习，激励具有高候选间ROUGE分数的输出。在XSum和CNN/DM数据集上的实验表明，相比BRIO，我们的方法在一致性G-Eval得分上分别提升了6.25%和3.82%。
原文摘要 · Abstract (English)
Hallucination refers to the inaccurate, irrelevant, and inconsistent text generated from large language models (LLMs). While the LLMs have shown great promise in a variety of tasks, the issue of hallucination still remains a major challenge for many practical uses. In this paper, we tackle the issue of hallucination in abstract text summarization by mitigating exposure bias. Existing models targeted for exposure bias mitigation, namely BRIO, aim for better summarization quality in the ROUGE score. We propose a model that uses a similar exposure bias mitigation strategy but with a goal that is aligned with less hallucination. We conjecture that among a group of candidate outputs, ones with hallucinations will comprise the minority of the whole group. That is, candidates with less similarity with others will have a higher chance of containing hallucinated content. Our method uses this aspect and utilizes contrastive learning, incentivizing candidates with high inter-candidate ROUGE scores. We performed experiments on the XSum and CNN/DM summarization datasets, and our method showed 6.25% and 3.82% improvement, respectively, on the consistency G-Eval score over BRIO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。