通过动态调整注意力图，有效减少大模型生成中的幻觉问题。
Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation
- 用训练好的分类器识别易引发幻觉的注意力分布，进行针对性干预。
- 在XSum摘要任务中幻觉率降低10%，计算效率提升7倍。
- 无需重新训练模型，适合快速部署到各类开源大模型中。
在摘要和开放书问答等任务中，大型语言模型常出现‘上下文幻觉’，即虽有准确信息却仍生成无关或错误内容，这通常源于模型过度依赖自生成内容而忽略输入上下文。为此，本文提出一种新方法‘引导注意力图编辑’（GAME），在推理阶段动态调整注意力权重以增强上下文相关性。GAME利用训练好的分类器识别易导致幻觉的注意力图，并基于梯度引导的‘编辑方向’，对不同注意力头的权重进行精准重分配，从而有效抑制幻觉。在具有挑战性的摘要和开放书问答任务上的全面评估表明，GAME能持续降低幻觉率。具体而言，在XSum摘要任务中幻觉率下降10%，同时相比当前最优基线实现7倍计算效率提升。
原文摘要 · Abstract (English)
In tasks like summarization and open-book question answering (QA), Large Language Models (LLMs) often encounter "contextual hallucination", where they produce irrelevant or incorrect responses despite having access to accurate source information. This typically occurs because these models tend to prioritize self-generated content over the input context, causing them to disregard pertinent details. To address this challenge, we introduce a novel method called "Guided Attention Map Editing" (GAME), which dynamically adjusts attention maps to improve contextual relevance. During inference, GAME employs a trained classifier to identify attention maps prone to inducing hallucinations and executes targeted interventions. These interventions, guided by gradient-informed "edit directions'', strategically redistribute attention weights across various heads to effectively reduce hallucination. Comprehensive evaluations on challenging summarization and open-book QA tasks show that GAME consistently reduces hallucinations across a variety of open-source models. Specifically, GAME reduces hallucinations by 10% in the XSum summarization task while achieving a 7X speed-up in computational efficiency compared to the state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。