通过动态关注机制提升大模型对上下文的忠实度，减少幻觉。
Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models
- 利用注意力分布与不确定度信号，在单次解码中动态引导上下文使用。
- 在开放书问答数据集上显著提升输出忠实度，且不增加计算开销。
- 适合需要高可信度生成的场景，如医疗、法律等专业领域。
大语言模型常因上下文整合不充分而产生与检索信息偏离的上下文忠实性幻觉。我们的分析发现，词元级不确定性与幻觉存在强相关性。基于此，我们提出轻量级框架DAGCD，通过单一解码过程利用注意力分布和不确定性信号动态引导上下文使用。在开放书问答数据集上的实验表明，DAGCD在保持计算效率的同时，显著提升了生成结果的忠实度与鲁棒性。
原文摘要 · Abstract (English)
Large language models (LLMs) often exhibit Context Faithfulness Hallucinations, where outputs deviate from retrieved information due to incomplete context integration. Our analysis reveals a strong correlation between token-level uncertainty and hallucinations. We hypothesize that attention mechanisms inherently encode context utilization signals, supported by probing analysis. Based on these insights, we propose Dynamic Attention-Guided Context Decoding (DAGCD), a lightweight framework that leverages attention distributions and uncertainty signals in a single-pass decoding. Experiments on open-book QA datasets demonstrate DAGCD's effectiveness, yielding significant improvements in faithfulness and robustness while preserving computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。