发现视觉语言模型推理时注意力如脉冲般散乱，提出无需训练的方法提升多图理解准确率。
Decoding the Pulse of Reasoning VLMs in Multi-Image Understanding Tasks
- 将推理过程拆分为规划与聚焦交替的模块，用软门控控制注意力
- 在BLINK基准上提升3.7%，MuirBench上提升1.07%
- 适用于希望不改模型即可增强推理能力的研究者
多图像推理仍是视觉语言模型（VLMs）的重大挑战。我们研究了一种此前被忽视的现象：在思维链（CoT）生成过程中，推理型VLM的文本到图像（T2I）注意力呈现出分散的“脉冲”特征——即间歇性、无焦点的注意力模式，无法集中于任务相关图像。我们进一步揭示了图像间注意力分配存在系统性位置偏差。受此启发，我们提出PulseFocus，一种无需训练、仅在推理阶段使用的方法，将CoT推理结构化为交替的计划/聚焦块，并引入软注意力门控机制。通过强制模型显式规划应关注的图像，并在解码阶段限制注意力至该图像，PulseFocus显著提升了注意力聚焦程度，在多图像基准测试中表现稳定：在BLINK基准上提升3.7%，在MuirBench上提升1.07%。
原文摘要 · Abstract (English)
Multi-image reasoning remains a significant challenge for vision-language models (VLMs). We investigate a previously overlooked phenomenon: during chain-of-thought (CoT) generation, the text-to-image (T2I) attention of reasoning VLMs exhibits diffuse "pulses": sporadic and unfocused attention patterns that fail to concentrate on task-relevant images. We further reveal a systematic positional bias in attention allocation across images. Motivated by these observations, we propose PulseFocus, a training-free, inference-time method that structures CoT reasoning into interleaved plan/focus blocks with soft attention gating. By forcing the model to explicitly plan which image to examine and then gating decode-time attention to the referenced image, PulseFocus sharpens attention focus and yields consistent improvements on multi-image benchmarks like BLINK benchmark (+3.7%) and MuirBench (+1.07%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。