arXiv:2602.04413cs.CLcs.AI2026-02

让视觉模型像人一样反复检查并修正错误,提升问答准确率。

History-Guided Iterative Visual Reasoning with Self-Correction

  • 通过多次看图并参考历史答案动态纠错
  • 在ScienceQA上达到78.9%准确率,比基线高107%
  • 适合需要高可靠推理的跨模态任务

自一致性方法是提升多模态大模型推理可靠性的重要技术。通过重复采样生成多个推理结果,并以投票方式选择最优答案,该方法在跨模态任务中表现突出。然而,现有方法多局限于固定的‘重复采样与投票’范式,未能复用历史推理信息,导致模型难以主动纠正视觉理解错误,也无法在迭代过程中动态调整推理路径。受人类反复验证与动态纠错行为启发,本文提出H-GIVR框架。在迭代推理中,多模态大模型多次观察图像,并将先前生成的答案作为后续步骤的参考,实现错误的动态修正,从而提升答案准确性。我们在五个数据集和三个模型上进行了全面实验。结果表明,H-GIVR框架显著提升了跨模态推理准确率,同时保持低计算开销。例如,在ScienceQA数据集上使用 exttt{Llama3.2-vision:11b}模型,平均每个问题只需2.57次响应即可达到78.90%的准确率,相比基线提升107%。

原文摘要 · Abstract (English)

Self-consistency methods are the core technique for improving the reasoning reliability of multimodal large language models (MLLMs). By generating multiple reasoning results through repeated sampling and selecting the best answer via voting, they play an important role in cross-modal tasks. However, most existing self-consistency methods are limited to a fixed ``repeated sampling and voting'' paradigm and do not reuse historical reasoning information. As a result, models struggle to actively correct visual understanding errors and dynamically adjust their reasoning during iteration. Inspired by the human reasoning behavior of repeated verification and dynamic error correction, we propose the H-GIVR framework. During iterative reasoning, the MLLM observes the image multiple times and uses previously generated answers as references for subsequent steps, enabling dynamic correction of errors and improving answer accuracy. We conduct comprehensive experiments on five datasets and three models. The results show that the H-GIVR framework can significantly improve cross-modal reasoning accuracy while maintaining low computational cost. For instance, using \texttt{Llama3.2-vision:11b} on the ScienceQA dataset, the model requires an average of 2.57 responses per question to achieve an accuracy of 78.90\%, representing a 107\% improvement over the baseline.

多模态推理自洽性视觉理解动态纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。