解决大模型多图理解时信息混杂问题，提升准确率。
Mitigating Cross-Image Information Leakage in Multi-Image Understanding with Large Vision-Language Models
- 通过随机噪声掩码单张图像，引导模型聚焦当前图
- 在多图任务上显著提升多个基准测试表现
- 无需训练或改架构，适用于视频理解等场景
大视觉语言模型（LVLMs）在单图任务上表现优异，但在处理多图输入时性能明显下降。我们发现不同图像的视觉元素会在模型表示和响应中发生纠缠，称为跨图信息泄露。为此，我们提出FOCUS方法：在训练无关、架构无感的前提下，每次仅保留一张图像，其余用噪声掩码，使模型聚焦于单一清晰图像；通过多轮遮蔽获取预测分数，再用纯噪声输入作为参考进行优化，抑制信息泄露，提升输出准确性。FOCUS在多个多图基准测试中持续提升性能，并可泛化至视频理解任务，证明其是一种无需额外训练或修改结构的通用增强方案。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) exhibit strong performance on single-image tasks. However, their performance degrades significantly when handling multi-image inputs. While this degradation has been observed in prior work, its nature remains poorly understood. We empirically observe visual elements from different images become entangled in the model's representations and responses. We refer to this phenomenon as cross-image information leakage. To address this issue, we propose FOCUS, a training-free and architecture-agnostic method. FOCUS masks all but one image with random noise, guiding the model to focus on the single clean image. This process is applied across the target images to obtain logits under partially masked contexts. These logits are aggregated and then refined using a noise-only reference input, which suppresses the leakage and yields more accurate outputs. FOCUS consistently improves performance on diverse multi-image benchmarks. We further show that FOCUS generalizes to video understanding, extending its applicability beyond static multi-image inputs. This demonstrates that FOCUS offers a general solution for enhancing multi-image reasoning without additional training or architectural modifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。