为多模态检索增强生成设计视觉查询预处理基准，提升真实场景下图像查询的准确性。
Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation
- 将视觉查询修复建模为大模型自主诊断并调用工具的智能决策任务
- 46,700个不完美查询测试显示，视觉缺陷使检索召回率大幅下降
- 监督微调可让小模型性能媲美大模型，助力构建鲁棒多模态系统
多模态检索增强生成（MRAG）已成为将外部知识引入多模态大语言模型（MLLMs）的关键范式。尽管文本RAG中查询预处理（如重写）已是标准流程，现有MRAG管道大多将视觉输入视为静态且不可变，隐含假设其无噪声。然而真实世界的视觉查询常存在几何失真、质量退化或语义模糊等问题，导致检索失败。为此，我们提出V-QPP-Bench，首个专注于视觉查询预处理（V-QPP）的综合性基准。我们将V-QPP建模为代理式决策任务，要求MLLM自主诊断缺陷并部署感知工具进行查询优化。在46,700个不完美查询及多种MRAG范式上的评估揭示三个关键发现：(1) 脆弱性——视觉缺陷严重降低检索召回率与端到端MRAG性能；(2) 修复潜力与瓶颈——虽理想预处理可恢复近乎完美的性能，但现成的MLLM在工具选择与参数预测上表现不佳，需专门训练；(3) 训练增益——监督微调使紧凑模型达到甚至超过大型专有模型的性能，验证了该基准对构建鲁棒MRAG系统的价值。代码已开源。
原文摘要 · Abstract (English)
Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a key paradigm for grounding MLLMs with external knowledge. While query pre-processing (e.g., rewriting) is standard in text-based RAG, existing MRAG pipelines predominantly treat visual inputs as static and immutable, implicitly assuming they are noise-free. However, real-world visual queries are often ``imperfect'' -- suffering from geometric distortions, quality degradation, or semantic ambiguity -- leading to catastrophic retrieval failures. To address this gap, we propose V-QPP-Bench, the first comprehensive benchmark dedicated to Visual Query Pre-processing (V-QPP). We formulate V-QPP as an agentic decision-making task where MLLMs must autonomously diagnose imperfections and deploy perceptual tools to refine queries. Our extensive evaluation across 46,700 imperfect queries and diverse MRAG paradigms reveals three critical insights: (1) Vulnerability -- visual imperfections severely degrade both retrieval recall and end-to-end MRAG performance; (2) Restoration Potential \& Bottleneck -- while oracle preprocessing recovers near-perfect performance, off-the-shelf MLLMs struggle with tool selection and parameter prediction without specialized training; and (3) Training Enhancement -- supervised fine-tuning enables compact models to achieve comparable or superior performance to larger proprietary models, demonstrating the benchmark's value for developing robust MRAG systems The code is available at https://github.com/phycholosogy/VQQP_Bench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。