根据问题复杂度动态调整文档页数，提升视觉问答效率与准确率。
ViSAR: Training-Free Adaptive-$k$ Retrieval for Visual Document Question Answering
- 直接在嵌入空间构建查询相关的页面相似矩阵，实现无需训练的自适应检索。
- 相比固定检索数量，最多降低58.7%的推理延迟，且保持或提升答案准确率。
- 相似矩阵结构可预测答案质量，为后续检索质量感知系统提供新方向。
文档视觉问答（DocVQA）常采用检索增强生成（RAG）框架，使用晚期交互编码器识别与用户查询相关的文档页面，再由大视觉语言模型（LVLM）生成答案。现有方法通常对所有查询固定检索前k页，导致LVLM延迟增加且可能降低答案准确率。本文提出ViSAR（视觉语义激活检索），一种无需训练的自适应k值检索方法，用于晚期交互式视觉文档检索。ViSAR在嵌入空间中直接构建查询相关的页面级相似矩阵，突出查询相关语义，并动态确定应检索的页面数量。在多个编码器和LVLM上验证，ViSAR能检索紧凑且查询适配的页面集合，使RAG延迟最高降低58.7%，同时维持或优于固定top-k及自适应启发式方法的答案准确率。此外，我们发现相似矩阵结构与答案准确率相关，提示未来可发展基于检索质量感知的文档理解方向。
原文摘要 · Abstract (English)
Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k$ number of pages regardless of query complexity, which increases LVLM latency and may degrade answer accuracy. We introduce ViSAR (Visual Semantic Activation Retrieval), a training-free adaptive-$k$ retrieval method for late-interaction visual document retrieval. ViSAR operates directly in the embedding space to construct a query-conditioned page-level similarity matrix that highlights query-relevant semantics and dynamically determines the number of pages to retrieve. Across multiple encoders and LVLMs, ViSAR retrieves compact, query-adapted page sets that reduce RAG latency by up to 58.7\%, while maintaining or improving answer accuracy compared with fixed top-$k$ and adaptive retrieval heuristics. Furthermore, we show that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。