让大模型高效看懂高清图像,通过智能切换搜索策略解决漏检和冗余难题。
CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception

- 根据全局信息动态切换专家搜索与语义感知扫描,避免盲区和冗余
- 在多个高清图像基准上达到顶尖准确率,搜索效率显著提升
- 适合需要精准处理高分辨率视觉输入的研究者与开发者
高分辨率图像理解是多模态大语言模型的关键瓶颈。尽管视觉搜索提供潜在解决方案,现有方法在覆盖范围与效率间难以平衡:专家辅助搜索高效但易遗漏,扫描式搜索虽能全覆盖却存在计算冗余和语义碎片化问题。为此,我们提出CVSearch,一种无需训练的自适应框架,通过评估-搜索工作流动态调度搜索策略。当全局信息不足时,先启用专家辅助搜索;失败后才触发新型语义感知扫描机制。不同于固定网格划分,该扫描范式采用语义引导的自适应补丁分割,有效缓解对象碎片化。此外,基于视觉复杂度先验设计的动态自底向上搜索策略,实现对局部细节的高效精确迭代探索。在多个高分辨率基准上的实验表明,CVSearch在保持顶尖准确率的同时大幅提升搜索效率。代码已公开于https://github.com/liliupeng28/ICML26-CVSearch。
原文摘要 · Abstract (English)
High-resolution (HR) image perception presents a key bottleneck for multimodal large language models (MLLMs). While visual search offers a promising solution, existing methods struggle with the trade-off between coverage and efficiency. Visual expert-assisted search is efficient but prone to blind spots when proposals fail, whereas scan-based search guarantees coverage at the cost of computational redundancy and semantic fragmentation. To address this dilemma, we introduce CVSearch, a training-free adaptive framework that dynamically schedules search strategies via an Assess-then-Search workflow. Specifically, CVSearch first invokes expert-assisted search when global information is insufficient, and only triggers a novel semantic-aware scanning mechanism upon failure. Distinct from rigid grid partitioning, this efficient scanning paradigm incorporates Semantic Guided Adaptive Patching to decompose images into semantically consistent regions, effectively mitigating object fragmentation. Furthermore, we devise a Dynamic Bottom-Up Search strategy driven by a Visual Complexity prior to enable efficient and precise iterative exploration of local details. Extensive experiments on HR benchmarks demonstrate that CVSearch achieves state-of-the-art accuracy while substantially improving search efficiency. Code is released at https://github.com/liliupeng28/ICML26-CVSearch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。