arXiv:2510.20812cs.CVcs.AI2025-10中稿 · ICLR被引 5

用小模型猜答案,大模型纠错,高效解决复杂图文推理问题。

Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculation

  • 多个小模型并行生成推理路径,提供多样定位候选。
  • 大模型整合高一致路径,准确率提升12.3%且计算成本低。
  • 无需训练,适合需要快速推理的高密度图文任务场景。

大型视觉语言模型在多模态理解上取得显著进展,但在密集图文混排、信息复杂的图像上推理能力受限,主要难点在于精准定位关键线索及跨步推理整合分散证据。本文提出无需训练的推测性裁决框架(Speculative Verdict, SV),借鉴推测解码思想,将多个轻量级草稿专家与一个强大的裁决模型结合。草稿阶段,多个小型VLM作为草稿专家生成多种推理路径,提供多样化定位候选;裁决阶段,大型VLM融合这些路径生成最终答案,同时引入共识专家选择机制,仅传递高一致性路径,显著降低计算开销。实验表明,SV在InfographicVQA、ChartMuseum、ChartQAPro和HR-Bench 4K等高难度信息密集型视觉问答数据集上持续提升性能,通过整合多个部分正确的推理路径实现错误纠正与成本效率平衡,优于大型专有模型或训练流程。代码已开源。

原文摘要 · Abstract (English)

Large Vision-Language Models (VLMs) have achieved remarkable progress in multimodal understanding, yet they struggle when reasoning over information-intensive images that densely interleave textual annotations with fine-grained graphical elements. The main challenges lie in precisely localizing critical cues in dense layouts and multi-hop reasoning to integrate dispersed evidence. We propose Speculative Verdict (SV), a training-free framework inspired by speculative decoding that combines multiple lightweight draft experts with a large verdict model. In the draft stage, small VLMs act as draft experts to generate reasoning paths that provide diverse localization candidates; in the verdict stage, a strong VLM synthesizes these paths to produce the final answer, minimizing computational cost while recovering correct answers. To further improve efficiency and accuracy, SV introduces a consensus expert selection mechanism that forwards only high-agreement reasoning paths to the verdict. Empirically, SV achieves consistent gains on challenging information-intensive and high-resolution visual question answering benchmarks, including InfographicVQA, ChartMuseum, ChartQAPro, and HR-Bench 4K. By synthesizing correct insights from multiple partially accurate reasoning paths, SV achieves both error correction and cost-efficiency compared to large proprietary models or training pipelines. Code is available at https://github.com/Tinaliu0123/speculative-verdict.

视觉推理多模态轻量模型高效架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。