arXiv:2603.03825cs.CVcs.AI2026-03被引 9

通过注意力引导冷启动,显著提升多模态推理模型性能。

From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning

  • 提出视觉注意力评分(VAS)量化模型对视觉信息的关注程度。
  • 发现文本冷启动可有效提升注意力,而多模态冷启动反而表现不佳。
  • 设计无需重训练的干预策略,性能提升1-2%,适合优化推理效率。

冷启动初始化在多模态大推理模型(MLRMs)训练中起关键作用,但其机制仍不清晰。我们引入视觉注意力分数(VAS),一种基于注意力的度量,用于量化模型对视觉标记的关注程度。发现推理性能与VAS高度相关(r=0.9616):VAS越高,多模态推理能力越强。令人惊讶的是,多模态冷启动未能提升VAS,注意力分布接近基础模型;而仅文本冷启动则带来明显提升,我们称之为“懒惰注意力定位”。为验证其因果作用,设计无需训练的干预方法,直接调节推理时注意力分配,实现1%–2%的性能增益。基于此,提出注意力引导的视觉锚定与反思(AVAR)框架,整合视觉锚定数据生成、注意力引导目标和视觉锚定奖励塑造。应用于Qwen2.5-VL-7B,在7个多模态推理基准上平均提升7.0%。消融实验进一步证实各组件贡献逐步递进。代码、数据与模型已在https://github.com/lrlbbzl/Qwen-AVAR公开。

原文摘要 · Abstract (English)

The cold-start initialization stage plays a pivotal role in training Multimodal Large Reasoning Models (MLRMs), yet its mechanisms remain insufficiently understood. To analyze this stage, we introduce the Visual Attention Score (VAS), an attention-based metric that quantifies how much a model attends to visual tokens. We find that reasoning performance is strongly correlated with VAS (r=0.9616): models with higher VAS achieve substantially stronger multimodal reasoning. Surprisingly, multimodal cold-start fails to elevate VAS, resulting in attention distributions close to the base model, whereas text-only cold-start leads to a clear increase. We term this counter-intuitive phenomenon Lazy Attention Localization. To validate its causal role, we design training-free interventions that directly modulate attention allocation during inference, performance gains of 1$-$2% without any retraining. Building on these insights, we further propose Attention-Guided Visual Anchoring and Reflection (AVAR), a comprehensive cold-start framework that integrates visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping. Applied to Qwen2.5-VL-7B, AVAR achieves an average gain of 7.0% across 7 multimodal reasoning benchmarks. Ablation studies further confirm that each component of AVAR contributes step-wise to the overall gains. The code, data, and models are available at https://github.com/lrlbbzl/Qwen-AVAR.

多模态推理注意力机制冷启动优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。