arXiv:2604.12890cs.CVcs.AI2026-04被引 6

解决长时序多模态搜索中信息爆炸和视觉丢失问题

Towards Long-horizon Agentic Multimodal Search

论文配图:Towards Long-horizon Agentic Multimodal Search
图 1 · 摘自论文原文
  • 用文件系统保存视觉数据,以文本标识符替代原始图像减少上下文负担
  • 支持100轮搜索,比现有开源模型在多个长时序任务上表现更优
  • 适合需要跨模态深度推理的长流程智能搜索场景

多模态深度搜索代理通过迭代收集文本和视觉证据,在解决复杂任务方面展现出巨大潜力。然而,长期任务中异构信息管理与高令牌成本仍是关键挑战,现有方法常因上下文爆炸或关键视觉信号丢失而受限。为此,我们提出一种新型长时序多模态深度搜索框架LMM-Searcher,核心为基于文件系统的视觉表征机制。通过将视觉资产外置于外部文件系统,并映射为轻量级文本标识符(UID),该方法有效缓解上下文开销,同时保留未来可访问的多模态信息。我们为代理配备了定制的取图工具,实现按需加载的渐进式视觉感知策略。此外,设计了一条数据合成流水线,生成需复杂跨模态多跳推理的查询,从中提炼出12,000条高质量轨迹,用于微调Qwen3-VL-Thinking-30A3B,构建专用多模态深度搜索代理。在四个基准上的大量实验表明,本方法成功扩展至100轮搜索,显著优于现有开源模型在MM-BrowseComp和MMSearch-Plus等挑战性长时序基准上的表现,且对不同基础模型具备良好泛化能力。代码将于https://github.com/RUCAIBox/LMM-Searcher发布。

原文摘要 · Abstract (English)

Multimodal deep search agents have shown great potential in solving complex tasks by iteratively collecting textual and visual evidence. However, managing the heterogeneous information and high token costs associated with multimodal inputs over long horizons remains a critical challenge, as existing methods often suffer from context explosion or the loss of crucial visual signals. To address this, we propose a novel Long-horizon MultiModal deep search framework, named LMM-Searcher, centered on a file-based visual representation mechanism. By offloading visual assets to an external file system and mapping them to lightweight textual identifiers (UIDs), our approach mitigates context overhead while preserving multimodal information for future access. We equip the agent with a tailored fetch-image tool, enabling a progressive, on-demand visual loading strategy for active perception. Furthermore, we introduce a data synthesis pipeline designed to generate queries requiring complex cross-modal multi-hop reasoning. Using this pipeline, we distill 12K high-quality trajectories to fine-tune Qwen3-VL-Thinking-30A3B into a specialized multimodal deep search agent. Extensive experiments across four benchmarks demonstrate that our method successfully scales to 100-turn search horizons, achieving state-of-the-art performance among open-source models on challenging long-horizon benchmarks like MM-BrowseComp and MMSearch-Plus, while also exhibiting strong generalizability across different base models. Our code will be released in https://github.com/RUCAIBox/LMM-Searcher.

多模态搜索长时序推理视觉表征智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。