发现让大模型专注关键信息的新方法，提升长文本理解能力
Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts
- 通过识别控制注意力的特殊头，定位影响模型聚焦的关键方向
- 在多个长文本任务中，启用关注方向后模型表现显著提升
- 无需指定具体相关段落，即可让模型自动关注重要信息，适合长文本场景
长上下文大语言模型容易被无关内容分散注意力，其原因尚不明确。本文首次识别出一类称为‘上下文头’的特殊注意力头，它们控制模型整体注意力分配。研究发现，注意力分散源于这些头未能充分关注相关上下文，可通过增强对相关部分的关注来缓解。进一步发现，在这些头的键和查询激活中存在‘关注方向’，能有效引导注意力集中于相关上下文，而无需显式指定相关内容。我们在多种长上下文任务中全面评估了关注方向的效果，结果表明该机制可有效改善长上下文模型的任务对齐问题。本研究为理解与优化长上下文大模型的注意力机制提供了新思路。
原文摘要 · Abstract (English)
Long-context large language models (LLMs) are prone to be distracted by irrelevant contexts. The reason for distraction remains poorly understood. In this paper, we first identify the contextual heads, a special group of attention heads that control the overall attention of the LLM. Then, we demonstrate that distraction arises when contextual heads fail to allocate sufficient attention to relevant contexts and can be mitigated by increasing attention to these contexts. We further identify focus directions, located at the key and query activations of these heads, which enable them to allocate more attention to relevant contexts without explicitly specifying which context is relevant. We comprehensively evaluate the effect of focus direction on various long-context tasks and find out focus directions could help to mitigate the poor task alignment of the long-context LLMs. We believe our findings could promote further research on long-context LLM alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。