通过自适应筛选文档页，大幅提升多页文档问答效率
AVIR: Adaptive Visual In-Document Retrieval for Efficient Multi-Page Document Question Answering
- 先用轻量模型评分每页相关性，再聚类筛选关键页
- 平均减少70%所需页面数，答对率84.58%且成本更低
- 无需微调大模型，适合资源受限的文档问答场景
多页文档视觉问答（MP-DocVQA）因长文档导致计算资源压力大、注意力机制失效而面临挑战。本文提出自适应视觉文档内检索框架（AVIR）：首先用轻量检索模型评估各页与问题的相关性，根据得分分布进行聚类，自适应选择相关内容；再通过Top-K筛选保持上下文紧凑。对于短文档，因聚类可靠性下降，改用相关性概率阈值进行页面选择。最终仅将筛选后的页面输入冻结的大型视觉语言模型（LVLM）生成答案，无需模型微调。该方法在MP-DocVQA数据集上将平均问答页数减少70%，达到84.58%的ANLS分数，优于现有方法且计算成本显著降低。AVIR在SlideVQA和DUDE基准上也验证了有效性。代码已开源。
原文摘要 · Abstract (English)
Multi-page Document Visual Question Answering (MP-DocVQA) remains challenging because long documents not only strain computational resources but also reduce the effectiveness of the attention mechanism in large vision-language models (LVLMs). We tackle these issues with an Adaptive Visual In-document Retrieval (AVIR) framework. A lightweight retrieval model first scores each page for question relevance. Pages are then clustered according to the score distribution to adaptively select relevant content. The clustered pages are screened again by Top-K to keep the context compact. However, for short documents, clustering reliability decreases, so we use a relevance probability threshold to select pages. The selected pages alone are fed to a frozen LVLM for answer generation, eliminating the need for model fine-tuning. The proposed AVIR framework reduces the average page count required for question answering by 70%, while achieving an ANLS of 84.58% on the MP-DocVQA dataset-surpassing previous methods with significantly lower computational cost. The effectiveness of the proposed AVIR is also verified on the SlideVQA and DUDE benchmarks. The code is available at https://github.com/Li-yachuan/AVIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。