主动搜索与修正感知,提升多模态大模型在高分辨率图像中的精确定位能力
ActiveScope: Actively Seeking and Correcting Perception for MLLMs

- 通过细粒度语义锚点独立定位目标,缓解语义偏差
- 抑制显著干扰项注意力,克服上下文主导问题,准确率达96.34%
- 无需训练,适合需要高精度视觉定位的科研与工业应用
多模态大语言模型(MLLMs)在视觉-语言理解方面表现优异,但在高分辨率图像中仍面临细粒度感知挑战。现有无训练方法多依赖基于注意力的定位或粗到精搜索,易受干扰项误导且难以定位多个目标。我们发现失败根源在于‘上下文主导’(Contextual Dominance)——显著干扰项压倒目标注意力;以及‘语义偏差’(Semantic Bias)——全局语义使模型聚焦最显著概念,导致多目标场景下定位不完整。基于此,我们提出ActiveScope,一种无训练框架,通过主动搜索与自我修正增强感知。其包含两个模块:语义锚点定位(SAL)利用细粒度语义锚点独立定位关键目标,缓解语义偏差;干扰抑制精炼(ISR)通过抑制显著干扰项注意力,克服上下文主导问题。在高分辨率图像理解基准测试中,ActiveScope显著优于现有无训练方法(如在$V^{*}$ Bench上达到96.34%准确率),验证了主动搜索与自校正范式的优越性。代码已开源。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have demonstrated impressive vision-language understanding, yet still struggle with fine-grained perception in high-resolution images. While existing training-free methods typically rely on attention-based localization or coarse-to-fine search, they are often misled by distractors and fail to locate multiple targets. Our investigation attributes these failures to Contextual Dominance, where salient distractors overwhelm target attention and cause inaccurate localization, and Semantic Bias, where global semantics cause the model to fixate on the most salient concept, resulting in incomplete localization in multi-object scenarios. Built on these insights, we propose ActiveScope, a training-free framework that enhances MLLMs by actively seeking and correcting perception. ActiveScope features two modules. The Semantic Anchor Localization (SAL) utilizes fine-grained semantic anchors to independently localize key targets, thereby mitigating semantic bias. The Interference-Suppressed Refinement (ISR) refines localization by suppressing attention on salient distractions to overcome contextual dominance. Extensive experiments on high-resolution image understanding benchmarks demonstrate that ActiveScope outperforms existing training-free methods (e.g., 96.34 percent accuracy on $V^{*}$ Bench), validating the superiority of the active search and self-correction paradigm. Our code is available at https://github.com/jasmine-ww/ActiveScope.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。