针对多模态大模型推理效率低的问题，提出自适应计算分配方法。
CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models
- 根据样本难易度动态分配计算资源，避免简单任务浪费算力。
- 在多个数据集上显著提升推理准确率，同时降低计算开销。
- 适合追求高效可靠推理的多模态应用开发人员使用。
多模态大语言模型（MLLMs）在视觉-语言任务中展现出强大的推理能力，但仍面临计算开销与任务难度不匹配的问题。通过实证分析发现，现有解码方法对简单样本过度计算，而对困难样本支持不足，影响模型效果与效率。本文首先建立采样覆盖、实例难度与残余风险之间的理论框架，揭示多模态推理存在长尾难度分布：少数困难或模糊样本主导了残余失败概率。基于此，提出覆盖感知多模态解码（CAMD），一种自适应推理机制，根据估计的不确定性动态分配计算。CAMD融合证据加权评分、后验覆盖估计与序贯贝叶斯更新，在有限的词元预算下实现效率与可靠性平衡。在多个基准数据集和基线上的实验验证了该方法的有效性与优势。
原文摘要 · Abstract (English)
Recent advances in Multimodal Large Language Models (MLLMs) have shown impressive reasoning capabilities across vision-language tasks, yet still face the challenge of compute-difficulty mismatch. Through empirical analyses, we identify that existing decoding methods may waste compute on easy cases while underserving hard ones, affecting both model effectiveness and efficiency. To address this issue, we first develop a theoretical framework that links sampling coverage, instance difficulty, and residual risk. Our analysis reveals that multimodal reasoning exhibits a heavy-tailed difficulty distribution; a small subset of hard or ambiguous samples dominates the residual failure probability. Based on this insight, we propose Coverage-Aware Multimodal Decoding (CAMD), an adaptive inference mechanism that dynamically allocates computation according to estimated uncertainty. CAMD integrates evidence-weighted scoring, posterior coverage estimation, and sequential Bayesian updating to balance efficiency and reliability under a limited token budget. Experiments on various benchmark datasets and baselines demonstrate the effectiveness and advantages of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。