arXiv:2604.23443cs.CL2026-04

VQA任务中贪心解码比随机采样更优,因答案分布偏斜且不确定性源于视觉信息缺失。

Revisiting Greedy Decoding for Visual Question Answering: A Calibration Perspective

论文配图:Revisiting Greedy Decoding for Visual Question Answering: A Calibration Perspective
图 1 · 摘自论文原文
  • 从校准角度理论证明贪心解码在特定条件下最优
  • 多基准测试显示贪心解码优于随机采样,准确率提升显著
  • 提出适用于多模态推理的贪心解码改进方法,适合严谨的视觉问答场景

大型语言模型(LLMs)广泛采用随机采样策略以平衡输出连贯性与多样性,这些启发式方法常被直接迁移到多模态大模型(MLLMs)中,缺乏任务针对性。然而我们指出,随机解码在视觉问答(VQA)任务中可能次优。VQA是封闭式任务,答案分布呈头部集中特征,其不确定性通常为认知性(epistemic),源于视觉证据缺失或模糊,而非多种合理延续。本文从理论上形式化了模型校准与预测准确性的关系,推导出贪心解码最优的充分条件。大量实验表明,贪心解码在多个基准上均优于随机采样。此外,我们提出面向推理模型的贪心解码方法,在多模态推理场景中超越标准贪心解码和随机采样。整体结果警示不应盲目沿用LLM解码启发式于MLLM,并证明贪心解码可作为VQA的高效且强健的默认策略。

原文摘要 · Abstract (English)

Stochastic sampling strategies are widely adopted in large language models (LLMs) to balance output coherence and diversity. These heuristics are often inherited in Multimodal LLMs (MLLMs) without task-specific justification. However, we contend that stochastic decoding can be suboptimal for Visual Question Answering (VQA). VQA is a closed-ended task with head-heavy answer distributions where uncertainty is usually epistemic, arising from missing or ambiguous visual evidence rather than plausible continuations. In this work, we provide a theoretical formalization of the relationship between model calibration and predictive accuracy, and derive the sufficient conditions for greedy decoding optimality. Extensive experiments provide empirical evidence for the superiority of greedy decoding over stochastic sampling across multiple benchmarks. Furthermore, we propose Greedy Decoding for Reasoning Models, which outperforms both stochastic sampling and standard greedy decoding in multimodal reasoning scenarios. Overall, our results caution against naively inheriting LLMs decoding heuristics in MLLMs and demonstrate that greedy decoding can be an efficient yet strong default for VQA.

视觉问答贪心解码多模态推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。