让AI像人一样分步看文档,精准找答案
Question-Guided Evidence Acquisition for Multimodal Visual Question Answering

- 用小代理动态决定下一步该看哪里,而非一次看完
- 在DocVQA2026上达65.0%准确率,远超之前的40.0%
- 适合需要精细阅读文档的复杂问答任务
多模态大模型能‘看见’文档,却难以可靠‘读取’。小字、表格、视觉线索和版面结构仍会干扰其直接视觉推理,即使页面已进入模型上下文。现有文档视觉问答系统多采用固定感知:一次性编码页面,提问后直接回答。我们提出Q-Guide,一个小型代理,可解析问题,判断缺失证据,并调用特定工具获取——如读取文字、放大细节或定位区域。在DocVQA2026和Manga109数据集上,Q-Guide分别达到65.0%和32.4%准确率,显著优于直接提示(40.0%)和近期多代理系统。性能随感知预算提升,两到三轮刻意观察即可获得主要收益;关键在于精准引导注意力,而非复杂控制逻辑——添加规划器、路由或协作代理均无帮助。
原文摘要 · Abstract (English)
Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them up under direct visual inference, even when the page is already sitting in the model's context. Most document-VQA systems treat perception as fixed: they encode the page once, ask the question, and answer from whatever the model happened to extract in that single fast pass. We think document VQA needs slower, more deliberate perception: rather than answering from one fixed encoding, the model should spend a bit of extra compute at inference time working out what to look at next, and only then answer. We build this into \textbf{Q-Guide}, a small agent that reads a question, works out what evidence it is still missing, and calls targeted tool(s) to recover it---reading text where text is needed, zooming in where detail is needed, or grounding a region where position matters. On DocVQA2026 and Manga109, Q-Guide outperforms both direct prompting and recent multi-agent document systems ($65.0\%$ vs.\ $40.0\%$ on DocVQA2026, $32.4\%$ vs.\ $24.4\%$ on Manga109), and the improvement holds across three Claude backbones (Opus 4.6, Sonnet 4.6, and Opus 4.5). We find that accuracy scales with the perception budget---most of the gain appears within two to three deliberate rounds---and that the gain comes from directing perception to the right place, not from complex control logic: adding planners, routers, or multiple collaborating agents does not help.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。