让AI像人一样主动看图推理,实现更精准的跨模态搜索。
Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning

- 通过主动关注图像细节,动态收集视觉证据进行多跳推理。
- 在5个基准上达到领先水平,超越多个闭源模型。
- 专为真实网络环境设计,适合需要深度视觉理解的任务。
多模态大语言模型在视觉任务中表现优异,但在复杂开放场景下常缺乏事实依据。尽管近期方法尝试通过外部工具解决此问题,但以视觉为核心原生的搜索范式仍研究不足。现有方法主要依赖语义明确的简单图像和纯文本证据路径,限制了多跳、跨模态推理与搜索能力。为此,我们提出 Visual-Seeker,一种基于主动视觉推理的视觉原生多模态深度搜索代理。不同于将视觉视为静态输入,该代理主动关注细粒度视觉特征,在搜索过程中动态获取视觉证据。为释放其视觉原生潜力,我们构建了主动视觉推理数据流水线,合成5000条高质量多模态轨迹用于模型训练。大量实验表明,该模型在5个具有挑战性的多模态搜索基准上均达领先性能,甚至超越多个闭源模型,验证了其在真实网络环境中稳健的视觉原生推理与搜索能力。代码与数据见:https://github.com/ZhengboZhang/Visual-Seeker。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have demonstrated impressive capabilities in many visual tasks, but they often struggle with factual grounding when confronted with complex, open-world scenarios. While recent multimodal deep search agents attempt to address this issue by utilizing external tools, the visual-native search paradigm remains underexplored. Existing methods primarily rely on simple images with explicit semantics and text-only evidence trajectories, limiting the agent's ability to perform multi-hop, cross-modal reasoning and search. To address these limitations, we propose Visual-Seeker, a visual-native multimodal deep search agent via active visual reasoning. Rather than treating vision as a static input, our agent actively attends to fine-grained visual details, dynamically harvests visual evidence throughout the search process. To unlock its visual-native potential, we design an active visual reasoning data pipeline and synthesize 5K high-quality multimodal trajectories for model training. Extensive experiments demonstrate the state-of-the-art performance across five challenging multimodal search benchmarks, even surpassing several proprietary models, validating robust visual-native reasoning and search in real-world web environments. The code and data can be accessed at: https://github.com/ZhengboZhang/Visual-Seeker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。