让AI回答复杂问题时能精准定位图文证据,看得见推理过程。
Chain of Evidence: Pixel-Level Visual Attribution for Iterative Retrieval-Augmented Generation

- 用视觉大模型直接分析文档截图,跳过繁琐的文本解析。
- 在需要理解图表布局的任务上,性能远超纯文本方法。
- 适合需要可解释性、依赖图文信息的问答系统开发者。
迭代检索增强生成(iRAG)通过逐步检索和推理外部文档,有效应对复杂的多跳问题。然而现有系统主要依赖解析后的文本,存在两大瓶颈:(1) 粗粒度归因,用户需手动在长文档中查找基于模糊文本引用的证据;(2) 视觉语义损失,将含图表的文档(如幻灯片、带图PDF)转为文本会丢失空间逻辑与版式线索,影响推理。为此,我们提出链式证据(Chain of Evidence, CoE),一种无需依赖特定检索器的视觉归因框架,利用视觉语言模型直接对检索到的文档截图进行推理。CoE 不依赖格式解析,输出精确的边界框,可视化完整的推理链条。我们在两个基准上评估: extbf{Wiki-CoE},基于2WikiMultiHopQA构建的大规模结构化网页数据集; extbf{SlideVQA},包含复杂图表和自由布局的幻灯片挑战数据集。实验表明,微调后的Qwen3-VL-8B-Instruct在需理解版式布局的场景中显著优于文本基线,同时实现像素级可解释的iRAG,且具有检索器无关性。代码已开源:https://github.com/PeiYangLiu/CoE.git。
原文摘要 · Abstract (English)
Iterative Retrieval-Augmented Generation (iRAG) has emerged as a powerful paradigm for answering complex multi-hop questions by progressively retrieving and reasoning over external documents. However, current systems predominantly operate on parsed text, which creates two critical bottlenecks: (1) \textit{Coarse-grained attribution}, where users are burdened with manually locating evidence within lengthy documents based on vague text-level citations; and (2) \textit{Visual semantic loss}, where the conversion of visually rich documents (e.g., slides, PDFs with charts) into text discards spatial logic and layout cues essential for reasoning. To bridge this gap, we present \textbf{Chain of Evidence (CoE)}, a retriever-agnostic visual attribution framework that leverages Vision-Language Models to reason directly over screenshots of retrieved document candidates. CoE eliminates format-specific parsing and outputs precise bounding boxes, visualizing the complete reasoning chain within the retrieved candidate set. We evaluate CoE on two distinct benchmarks: \textbf{Wiki-CoE}, a large-scale dataset of structured web pages derived from 2WikiMultiHopQA, and \textbf{SlideVQA}, a challenging dataset of presentation slides featuring complex diagrams and free-form layouts. Experiments demonstrate that fine-tuned Qwen3-VL-8B-Instruct achieves robust performance, significantly outperforming text-based baselines in scenarios requiring visual layout understanding, while establishing a retriever-agnostic solution for pixel-level interpretable iRAG. Our code is available at https://github.com/PeiYangLiu/CoE.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。