arXiv:2607.24651cs.CVcs.CL2026-07

不用坐标也能准找证据,模型回答更可信

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

论文配图:Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
图 1 · 摘自论文原文
  • 用文字引用证据,再由系统定位区域,替代坐标输出
  • 证据召回率提升26至47点,幻觉率减半
  • 无需人工标注区域,适合长文档的高效训练

可靠的视觉文档理解要求模型将每个答案准确关联到支持它的证据区域。近期基准和系统采用坐标接口:模型输出标记证据区域的边界框坐标。在此接口下,即使答案正确,视觉语言模型也常错误定位区域,这种现象称为归属幻觉。我们在经验证的双语CiteVQA子集上,比较了坐标接口与仅输出文本的语义接口——模型直接引用证据原文,由多模态检索器根据版面分析器返回对应页面区域(表格和图像通过其标题或注释引用);该对比在六种开源视觉语言模型上重复进行。相较于坐标接口,证据召回率从最高8点提升至26至47点,幻觉率约减半,而答案质量基本不变。基于此结果,我们采用相同的引用-检索流程作为训练框架:由于长文档中区域级标签成本高昂,提出GRPO训练策略,奖励来自人类评判的正确答案及检索区域图像裁片,使80亿参数主干模型的严格归属准确率从22.4提升至33.8,实现无需区域标签的高质量证据归属。

原文摘要 · Abstract (English)

Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.

文档理解证据归属无坐标零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。