arXiv:2607.01707cs.CV2026-07

解决视觉大模型推理时注意力丢失问题,提升长期推理准确性。

LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression

论文配图:LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
图 1 · 摘自论文原文
  • 通过双奖励机制调控视觉注意力轨迹与分布。
  • 在8个基准数据集上显著优于现有方法。
  • 适合需要长序列视觉推理的场景如复杂问答、文档理解。

大型视觉语言模型(LVLMs)虽具备强大推理能力,但在长序列解码过程中易出现视觉遗忘,导致注意力逐步偏离视觉证据。现有方法多将其视为后期注意力衰减问题,或依赖启发式提醒与事后修正。通过系统性实证分析,我们发现性能下降主要由两个被忽视因素驱动:早期注意力衰减影响证据获取,以及注意力过度集中于无关的视觉‘沉没’标记。受此启发,我们提出LASER——一种后训练框架,通过调节推理过程中的视觉注意力轨迹与视觉标记间注意力分布来应对该问题。技术上,LASER引入两项互补奖励:视觉锚定奖励,促使模型在解码全程保持对语义显著视觉标记的关注;沉没抑制奖励,惩罚对非信息性视觉标记的过度关注。两者协同作用,既保留早期视觉锚定,又防止注意力坍缩至无意义区域。在八个基准数据集上的广泛实验表明,LASER持续优于强基线,验证了注意力感知训练对缓解视觉遗忘的有效性。

原文摘要 · Abstract (English)

Large vision-language models (LVLMs) exhibit strong reasoning ability but suffer from visual forgetting during long-horizon decoding, where attention progressively drifts away from visual evidence. Existing methods largely treat this issue as a late-stage attention decay problem or attempt to mitigate it through heuristic reminders or post-hoc attention lifting. Through systematic empirical analysis, we find that performance degradation under visual forgetting is largely driven by two overlooked factors: early-stage attention decay disrupts evidence acquisition, and attention concentration on a subset of task-irrelevant visual sink tokens. Motivated by these insights, we propose LASER, a post-training framework that regulates both the visual attention trajectory and intra-visual token attention distribution during reasoning. Technically, LASER introduces two complementary rewards: a Visual Grounding Reward, which encourages the model to maintain attention on semantically salient visual tokens throughout decoding, and a Sink Suppression Reward, which penalizes excessive attention concentration on visual sink tokens. Together, these rewards preserve early-stage grounding while preventing attention collapse onto uninformative regions. Extensive experiments on eight benchmark datasets demonstrate that LASER consistently outperforms strong baselines, validating attention-aware training as an effective remedy for visual forgetting.

视觉语言模型注意力机制推理增强后训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。